1 



(19) 




Europaisches Patentamt 
European Patent Office 
Office europten des brevets 



(11) 



EP 0 806 873 A2 



(12) 



EUROPEAN PATENT APPLICATION 





Date of publication: 






12.11.1997 Bulletin 1997/46 


(21) 


Application number: 97107526.2 




(22) 


Date of filing: 07.05.1997 




(84) 


Designated Contracting States: 


(72) Inventors: 




DE FR GB 


• Kurobe,Akio 


(30) 




Taondabayashl-shi, Osaka-fu (JP) 


Priority: 08.05.1996 JP 113446/96 


• Shinoda, Mayumi 




29.05.1996 JP 134643/96 


Hirakata-shI, Osaka-fu (JP) 




17.07.1996 JP 187153/96 


• Ikeda, Kojl 




25.11.1996 JP 313266/96 


Neyagawa-shI, Osaka-fu (JP) 




20.01 .1997 JP 7275/97 


(71) 




(74) Representative: 


Applicant: 


Attenburg, Udo, Dipl.-Phys. et al 




MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD. 


Patent- und Rechtsanwdlte 




I^doma-shl, Osaka-fu (JP) 


Bardehle . Pagenberg . Dost . AHenburg . 






Frohwrtter . Gelssler & Partner, 






Postfach86 06 20 






81633 MQnchen (DE) 



CM 
< 
CO 

1^ 

CO 

o 

00 

o 

GL 
LU 



(54) Multiplex transmission method and system, and audio Jitter absorbing metliod used therein 




(57) A multiplex frame Is of a fixed length and 
includes a header and a fixed-length field. The fixed- 
length field is divided into variable-length slots 1 and 2. 
The length of the variable-length slot 1 is a predeter- 
mined fixed length when data to be stored exists and is 
zero with no data to be stored. The length of the varia- 
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ble-length slot 2 is incrgased /deaea 

data not employing reti^ismi^loh^{^e!q. . an audio 
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frame) is transmitted in the first variat)le-length slot and 
one complete variable-length data employing retrans- 
mission (e.g., a data link frame of video) is transmitted 
in the second variable-length slot. When the multiplex 
frame thus has a fixed length and the frame length of the 
data link frame of the variable-length data employing 
retransmission is changed depending on the length of 
the fixed-length data not employing retransmission, the 
transmitting timing of the data link frames can be fixed 
with respect to the transmission stream of the multiplex 
frames. Then the boundaries between the data link 
frames are not lost due to a transmission error, which 
improves the error robustness. That is to say, even if the 
slot synchronization is lost due to a transmission error, it 
Is possible to see the next header at once, providing 
easy resynchronization. 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention s 

The present invention relates to multiplex transmis- 
sion methods and systems, and to audio jitter absorbing 
methods used therein, and more particularly to a 
method of multiplexing and transmitting low-bitrate- 
compressed audio data and video data and/or computer 
data and a method of absorbing audio jitter caused in 
the transmission. 

Desaiption of the Background Art 

In visual telephones and video conference sys- 
tems, video signals having enormous amounts of infor- 
mation are generally tow-bitrate-coded and then 
transmitted to the destination. H.263 in the draft stage of 
the international standardization by ITU*T (International 
Telecommunication Union - Telecommunication Stand- 
ardization Sector) is known as an example of such low 
bit rate coding. These standardizations adopt the 
motion compensative interframe predictive coding and 
the variable-length coding as a compression coding 
method, which are very sensitive to transmission errors. 
That is to say. when an error, even of one bit, occurs, the 
Influence of the error transfers in a large area of a pic- 
ture and also transfers in the time axis direction, provid- 
ing serious deterioration of the image quality. Hence, in 
the analogue telephone line which is prone to transmis- 
sion errors, error confection by retransmission control is 
generally performed before image decoding. 

On the otiier hand, as shown in the draft G.723 
issued by ITU-T, audio data are generally compressed 
to a very low bit rate by low bitrate coding when trans- 
mitted. In the audio data, transmission errors are not so 
noticeable as in tiie video data. However, it is sensitive 
to tiie ti^nsmission delay. Hence, according to G.723, 
when an error is detected, muting is performed without 
error correction by retransmission, so as to prevent the 
transmission enror from causing no'^e. 

As a metiiod for multiplexing and transmitting video 
data, or variable-length data employing retransmission, 
and audio data, or fixed-length data not employing 
retransmission, ITU-T has also issued draft H.223. 

H.223 will now be explained referring to the draw- 
ings as an example of a conventional multiplex trans- 
mission method. 

Fig.43 shows a general layer structure of a multi- 
plex transmission device adopting tiie multiplex trans- 
mission method determined by H.223. In Fig.43. this 
multiplex transmission device includes a physical layer 
801, a multiplex layer 802. an adaptation layer 803. a 
video coder 804. an audio coder 805, a data protocol 
806. an LAPM 807. an H.245 control 808. a video I/O 
809, an audio I/O 810, and an application layer 811. 

H.223 supposes a modem for analogue telephone 
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line at 28.8 Kbps as the physical layer 801 . It defines, as 
the multiplex layer 802 and tiie adaptation layer 803, a 
frame structure and a procedure for video error con'ec- 
tion by retransmission control and a frame structure for 
audio error detection. Supposed as the video coder 804 
is H.263. As the audio coder 805. G.723 is supposed. 
tTU-T issues H.324 as a specification recommendation 
for the entire system. H.223 does not specifically define 
tiie data protocol 806. A communication procedure for 
control data is defined as the LAPM 807. /\s the H.245 
control 808, commands and procedure for system con- 
trol are specified. 

Next. Fig. 44 shows tiie multiplex frame format in the 
multiplex layer 802 of H.223. In Fig.44, for the opening 
flag and the closing flag, the same bit pattern 
"011 1 1 1 1 0" as tiiat used in HDLC is used to establish 
flag synchronization. In order to keep the transparency 
of the opening flag and the closing flag, H.223 defines to 
insert 0 after five contiguous I's in a part other tiian a 
flag and to remove it on the receiving end. The header 
shows, into which types of slots the following informa- 
tion field is divided and which types of data are multi- 
plexed. Fig.45 shows an exannple of a multiplex frame 
format in the muttiplex layer 802 with audio data, com- 
puter data and video data multiplexed. 

D^ined in the adaptation layer 803 are a frame for- 
mat and a procedure for error correction of video by 
retransmission control. Fig.46 shows an ARQ (Auto- 
matic Repeat reQuest) frame format for error conrectlon 
of video by retransmission control. In Fig.46. tiie control 
field contains a transmission number for retransmission 
control and a flag indicating whether the following pay- 
load field is an information frame or a supervisory 
frame. The payload field contains video data in the case 
of an information frame and contains a retransmission 
request number and a retransmission request com- 
mand in tiie case of a supervisory frame. The ORG field 
contains a check code for error detection. The proce- 
dure of retransmission contiol determined in the adeip- 
tation layer 803 is the selective repeat ARQ method of a 
window size of 128 frames, in which a retransmission is 
requested only once in a supervisory frame by using tiie 
tiBnsmission nurriber of a frame witii an error detected 
by the CRC check as the retransmission request 
number, together witii the retransmission request com- 
mand. 

The H.223 gives variable l»igths botii to multiplex 
frames and video frames and uses flag synchronization. 
Accordingly, frame synchronization is lost if a flag has 
an en^or When an en'or occurs in other data, that data 
may simulate itself as a flag to break the synchroniza- 
tion. The H.223 thus has tiie problem of being easily 
affected by error. 

The radio channel of the PHS (Personal Handy- 
phone System) which is a simplified portable telephone 
(which has a l:»t rate of 32 Kbps) is still more prone to 
transmission errors than the analogue telephone line. 
Conventionally, as a tiansmission control method used 
when transmitting computer data tiirough the radio 



15 



20 



25 



30 



35 



40 



45 



50 



2 



3 



EP0806 873A2 



4 



channel of PHS, PIAFS (PHS Internet Access Forum 
Standard) is suggested within the country of Japan, for 
example. 

An ARQ frame in PIAFS has a fixed length (80 
bytes). Fig.47 shows a frame structure of PIAFS. In 
Fig.47, the frame type identification field contains a 
frame type, the error control field contains a transmis- 
sion frame number and an oldest unreceived frame 
number, the user data length indication field contains 
the data length of significant data in the user data field, 
the user data field contains user data, and the error 
detection code field contains an error detection code for 
detecting en-or of the entire ARQ frame. 

In the PIAFS, frame synchronization is established 
prior to a communication by using a synchronizing 
frame including a synchronization flag and the frame 
synchronization is kept by using data frames having the 
same length as the synchronizing frame, which requires 
no synchronization flag in the data frames. The PIAFS, 
with fixed-length ARQ frames, is not susceptible to error 
because the frame syrx;hronization is not lost even if an 
error occurs. 

Hence, if fixed-length multiplex frames are used in 
H.223, the multiplex frames can be synchronized even if 
an error occurs, which will provide the multiplex frames 
with a high tolerance for en^ors. Japanese Patent Publi- 
cation No.8-13057 discloses an example thereof, as 
"Mixing Transfer System for HDLC Variable Length 
Packet and Non-HDLC Fixed Length Packet." The 
transfer method disclosed in the reference will now be 
explained refemng to Fig.48. 

In Fig.46, each fixed-length frame is provided with a 
header showing whether the contents are HDLC data or 
not. >A/hen the header indicates non-HDLC data, the 
fixed-length frame contains a non-HDLC fixed-length 
packet (ag., an audio data packet). When the header 
indicates HDLC data, the fixed-length frame contains 
part of a divided variable-length HDLC frame (ag., a 
video frame). According to this method, tiie multiplex 
frames, as having a fixed length provide error robust- 
ness. However, the video data still adopt the HDLC flag 
synchronization and are stored over a plurality of multi- 
plex frames. Accordingly, the video frames are stilt eas- 
ily affected by en^or. 

For the purpose of taking advantage of the function 
of increasing transmission t}and width for video data or 
the like by performing silence detection to produce no 
audio frame when silent, as in G.723, it is necessary to 
suppress jitter under several miri-seconds. This is due 
to the fact that if jitter is large, the receiving side can not 
tell whether audio data is not received due to jitter or 
audio data is not received because it is in a silent 
period. Then suitable processing can not be performed. 

Conventionally, as a method for minimizing jitter of 
fixed-length cell packets, a method of comparing a pre- 
determined threshold and the number of cells in a buffer 
having a capacity twice the maximum fluctuation time to 
control reading of the buffer is suggested. (Refer to Jap- 
anese Patent Laying-Open No.5-244186 "ATM/STM 



Converting Circuit.") Fig.49 shows the structure thereof. 
In Fig. 50, the buffer 50 has a capacity of 2At, twice the 
jitter At. The reading control portion 60 compares the 
number of cells accumulated in the buffer 50 and a pre- 

5 determined threshold. When the number of accumu- 
lated cells exceeds the threshold, it outputs a reading 
address to start reading the buffer 50. Thus, the jitter 
correcting circuit of Fig.49 is so structured as to read 
contents of the buffer 50 after a maximum delay time in 

10 the ATM network has passed after arrival of the first cell . 
However, when the jitter correcting circxiit of Fig.49 is 
used, the average delay time increases as the cells are 
accumulated to the threshold, inaeasing delay of audio. 
When the multiple header has an error, the frame 

15 number of the video frame multiplexed in the discarded 
information field can not be known and then a retrans- 
mission request can not be immediately transmitted to 
the transmitting side. Generally, the multiplex layer dis- 
cards data in the information field when an en^or is found 

20 in the multiplex header. Accordingly, it is when the 
sequence error is detected from the frame number of 
the ARQ header of the video frame received next with- 
out error that a retransmission request for the video 
frame in the discarded information f ieki is sent out from 

25 the error control processing layer. For exanrple, if a 
transmission error occurs in the multiplex header of the 
multiplex frame of the frame number (1) transmitted 
from the transmitting terminal, the receiving terminal 
discards video data in the multiplex frame after receiv- 

30 ing the multiplex header. Accordingly, the receiving ter- 
minal can not immediately send a reject for the video 
data of the frame number (1). That is to say, the receiv- 
ing terminal f irxls the absence of the multiplex frame of 
the frame number (1) only after it receives the multiplex 

35 franne of the frame number (2), and then it transmits a 
reject for the frame number (1) to the transmitting side. 

When the en-or detection for the frame numbers is 
perfbmied independently of the error detection for the 
information f iekis as shown in the partial repeat ARQ for 

40 mobile packet communications presented in Electronic 
Information Communication Society, National Meeting 
in Spring, B-323. 1990, the receiving side can, even if 
the information part is discarded, immediately transmit 
a retransmission request for the frame as long as the 

45 frame number of that frame correctly amves at the 
receiving side. This reduces the retransmission waiting 
time. 

SUMMARY OF THE INVENTION 

50 

A first object of the present invention is to provide a 
multiplex transmission method having en^or robustness 
and good transmission efficiency. 

A second object of the present invention is to pro- 
55 vide a multiplex transmission metfiod capable of cor- 
recting audio jitter without increasing delay time of 
audio. 

The present invention has the following features to 
achieve the objects above. 
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A first aspect of the present inventon is directed to 
a method for multiplexing ^TiJBnstruttiiig-vaHable:;^ 
|ength_data employing retransmission to which en-pr 
connection by retransmission control is applied and 
fi xed-length data not employing retransmission to which 
er ror correction b yjgransnr iission control is not applie d 
in multiplex frames, 

wherein the multiplex frames have a fixed length 
and a data link frame for staing the variable-length data 
enploying retransmission has a frame length 
increased/decreased deperxting on the length of the 
fixed-length data not employing retransmission to be 
multiplexed so that the data link frame is transmitted at 
timing kept in synchronization for each multiplex frame 
with the transmission stream of the multiplex frames. 

In accordance with the first aspect, the multiplex 
frames for storing the fixed-length data not employing 
retransmissbn and the variable-length data employing 
retransmission have a fixed length and a data link frame 
for storing the variable-length data employing retrans- 
mission has a frame length changed depending on the 
length of the fixed-length data not employing retrans- 
mission to be multiplexed so that the data link frames 
can be transmitted at timing synchronized with the 
transmission stream of the multiplex frames. Then the 
transmitting timing of the data link frames can be fixed 
and the boundaries between the data link frames are 
not lost in the event of a transmission error, providing 
error robustness. TTiat is to say. unlike H.223 which 
acquires synchronization with flags, bits are not shifted 
due to "0" insertion, and even if the slot synchronization 
is lost due to a transmission error, it is possible to see 
the next header at once, providing easy resynchroniza- 
tion. Moreover, it is possible to effectively utilize silent 
periods of the fixed-length data not employing retrar^- 
mission to increase the transmissioi efficiency of the 
vkleo data, or the variable-length data employing 
retransmission. 

According to a second aspect of the present inven- 
tion, in the first aspect, 

the multiplex frame includes a fixed-length field, 
the fixed-length field being divided into first and 
second variable-length slots, 
the first variable-length slot having a predetermined 
fixed length when fixed-length data not employing 
retransmission to be multiplexed exists and having 
a length of zero when no fixed-length data not 
employing retransmission to be multiplexed exists, 
the second variable*length slot having a relatively 
short, first lengtfi when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a relatively long, second length when no 
fixed-length data not employing retransmission to 
be multiplexed exists, 

wherein the fixed-length data not employing 
retransmission is transmitted in the first variable- 
length slot and the variable-length data employing 
retransmission is transmitted in the second varia- 



ble-length slot. 

According to a third aspect of the present invention, 
in the second aspect, 

5 

the multiplex frame further includes a nnultiplex 
frame header. 

the multiplex frame header containing multiplex 
information indicating whether the fixed-length data 
10 not employing retransmission is multiplexed in the 
multiplex frame. 

In accordance with the third aspect, it is possible to 
know the multiplex sti-ucture of ttie multiplex frames by 
15 seeing the multiplex information in tiie nnultiplex header. 
According to a fourth aspect of tiie present inven- 
tion, in the second aspect, 

the multiplex frame further includes a fixed-length 
20 slot for storing and transmitting tiie variable-length 
data employing retransmission. 

According to a fifth aspect of the present invention, 
in the fourth aspect, 

25 

variable-length data employing retransnnission of 
the same kind as the variable-length data employ- 
ing retransmission stored in the second variable- 
lengtii slot is stored in tiie fixed-length slot. 

30 

According to a sixtii aspect of the present invention, 
inthefif&i aspect. 

the lengtii of the fixed-length slot is selected equal 
35 to or larger than tiie length of tiie fixed-length field. 

In accordance with the sixth aspect, when variable- 
tengtii data employing retransmission ti^nsmrtted in tiie 
second variable-lengtii slot is en-ored, the variaWe- 
40 length data employing retransmission can be retrans- 
mitted in eitiier of the second variable-lengtii sbt and 
thefixed-lengtii slot. 

According to a seventh aspect of the present inven- 
tion, in the fourth aspect. 

45 

variable-length data employing retransmission of a 
different kind from tiie variable-length data employ- 
ing retransmission stored in the second variable- 
length slot is stored in tiie fixed-length slot. 

50 

In accordance witii the seventh aspect, it is possible 
to multiplex a plurality of kinds of variable-lengtii data 
employing retransmission. 

According to an eighth aspect of tiie present inven- 
55 tion, in the fourth aspect, 

the length of the data link frame for storing the vari- 
able-length data employing reti'ansmission is set 
equal to that of tiie second variable-length slot 
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when the variable-length data employing retrans- 
mission is transmitted in the second variable-length 
slot artd equal to that of the fixed-length slot when it 
is transmitted in the fixed-length slot. 

According to a ninth aspect of the present inven- 
tion, in the first aspect, 

the multiplex frame includes, 

a frame type identification field for storing a 
code for identifying the frame type, 
a user data field for storing user data, 
a user data length indication field for storing the 
lengith of significant data in the user data field, 
a retransmission control field for storing a 
frame number used in retransmission control, 
and 

an enror detecting code field for storing an error 
detecting code, 

wherein the user data field is divided 
into a field for the variable-length data employ- 
ing retransmission and a field for the fixed- 
length data not employing retransmission, 
the variable-length data employing retransmis- 
sion is stored in the field for variable-length 
data employing retransmission, 
thetixed-length data not employing retransmis- 
sion is stored in the field for fixed-length data 
not employing retransmission, 
the length of the variable-length data employ- 
ing retransmission stored in the field for varia- 
ble-length data employing retransmission 
accomodated in the user data length irviication 
field, and 

the error detecting code field contains an error 
detecting code for error detection in the part 
excluding the field for the fixed-length data not 
employing retransmission. 

In accordance with the ninth aspect of the present 
invention, it is possible to multiplex fixed-length data not 
employing retransmission and variable-length data 
employing retransmission using the structure of tiie 
ARQ frame in PIAFS almost as it is. 

According to a tenth aspect of the present inven- 
tion, in the ninth aspect, 



mission to be multiplexed exists, the variable-lengtii 
data employing retransmission is stored by using 
the entirety of the user data field as the field for var- 
iable-length data employing retransmission, 
5 wherein an identification code indicating 

whether the user data field includes the field for 
fixed-length data not employing retransmission is 
stored in the frame type identification field. 

10 According to an eleventh aspect of the present 
invention, in the first aspect, 

a basic nmltiplex frame stream including at least 
one first multiplex frame in which the fixed-length 
15 data not employing retransmission is multiplexed 
and at least one second multiplex frame in which 
the fixed-length data not employing retransmission 
is not multiplexed is repeatedly transmitted. 

so In accordance with tiie eleventh aspect, it is possi- 
ble to multiplex the fixed-length data not employing 
retransmission and tiie variable-length data employing 
retransmission in step witii the cyde of generation of tiie 
fixed-length data not employing retransmission. 

25 According to a twelftti aspect of tiie present inven- 
tion, in the eleventh aspect. 

tiie first multiplex frame includes a f ixed-lengtii field 
divided into first and second variable-length slots, 

30 tiie first variable-length slot having a predetermined 
fixed length when fixed-length data not employing 
retransmission to be multiplexed exists and having 
a lengUi of zero when no fixed-length data not 
employing retransmission to be multiplexed exists, 

35 tiie second variable-length slot having a relatively 
short, first length when fixed-lengtii data not 
employing retransmission to be multiplexed exists 
and having a relatively long, second lengtii when no 
ftxed-lengtii data not employing retransmission to 

40 be multiplexed exists, and 

tiie second multiplex frame includes a fixed-length 
slot 

wherein the fixed-length data not employing 
reti'ansmission is transmitted in the first variable- 
45 length slot and tiie variable-lengtii data employing 
retransmission is transmitted in tiie second varia- 
ble-length slot and tiie fixed-length slot. 



when fixed-length data not employing retransmis- 
sion to be multiplexed exists, tiie user data field is so 
divided into the field for variable-lengtii data 
employing retrar^ission and tiie field fa fixed- 
iengtti data not employing retransmission, the field 
for variable-length data employing retransmission 
storing tiie variable-length data employing retrans- 55 
mission and the field for fixed-length data not 
employing retransmission storing the fixed-lengtii 
data not employing reti'ansmission, and 
when no fixed-lengtii data not employing retrans- 



Aocording to a thirteemh aspect of tiie present 
invention, in tiie eleventii aspect. 

the first and second fixed-length multiplex frames 
each have a lengtti of 60 bytes, and 
the basic multiplex frame stream is formed of one of 
tiie first multiplex frame and one of the second 
fixed-lengtii multiplex frame, 

wherein one audio frame is stored as tiie 
fixed-lengtii data not enrptoying retransmission in 
tiie first multiplex frame. 
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Accx>rding to a fourteenth aspect of the present 
invention. In the eleventh aspect, 

the first and second fixed-length multiplex frames 
each have a length of 80 bytes, and 
the basic multiplex frame stream Is formed of two of 
the first multiplex frames and one of the second 
fixed-length multiplex frame, 

wherein the first multiplex frame contains 
one audio frame as the fixed-length data not 
employing retransmission. 

In accordance with the fourteenth aspect. It Is pos- 
sible to nrujltiplex audio data and other fixed-length data 
employing retransmission according to the structure of 
the ARQ frame of the PIAFS. 

According to a fifteenth aspect of the present inven- 
tion, In the eleventh aspect. 

the first and second fixed-length multiplex frames 
each have a length of 80 bytes, and 
the basic multiplex frame stream Is formed of one of 
the first nrujitlplex frame and two of the second 
fixed-length multiplex frames. 

wherein two audio frames are stored as the 
fixed-length data not employing retransmission In 
the first multiplex frame. 

In accordance with the fifteenth aspect, it Is possi- 
ble to multiplex audio data and other fixed-length data 
employing retransmission according to the structure of 
the ARQ frame In the PIAFS. 

A sixteenth aspect of the present Invention is 
directed to a method of absorbing audio jitter on the side 
receiving muttplex frames when a low-bitrate-coded N- 
byte (N is a positive Integer) audio frame generated at a 
rate of one frame per an F-byte (F Is a positive Integer 
and N<F) transmission time is stored and multiplexed in 
an M-byte (M Is a positive Integer and M<F) fixed-length 
multiplex frame and when at most J-byte (J is a positive 
integer) audio jitter is generated among the audio fields 
of the multiplex frames because F/M is not an integer, 

wherein a reference audio frame separated from 
the next audio frame at a minimum Interval Is identified 
from among the audio frames on the transmitted multi- 
plex frame stream, and 

using the timing at which the reference audio 
frame Is outputted as a reference, the following audio 
frames are outputted at the rate of one frame per the F- 
byte transmission time. 

In accordance with the sixteenth aspect, It Is possi- 
ble, on the side receiving the multiplex frames, to easily 
absorb audio jitter caused because the frame length of 
the multiplex frames and the transmission cycle of the 
audio frames do not match. 

According to a seventeenth aspect of the present 
invention, in the sixteenth aspect. 

different multiplex information are defined for n mul- 



tiplex frames containing multiplexed audio frames In 
a basic multiplex frame stream formed of m multi- 
plex frames, where n and m are integers and 
n • F=m • M , so that the reference audio frame 
5 can be Identified on the basis of the multiplex infor- 
mation. 

In accordance with the seventeenth aspect, differ- 
ent multiplex Information are defined for n multiplex 
10 frames containing audio frames multiplexed therein, 
which facilitates identification of the reference audio 
frame serving as a reference in the audio jitter at)sorp- 
tlon. 

According to an eighteenth aspect of the preserrt 
15 invention. In the sixteenth aspect. 

M=80 bytes and F=120 bytes. 

According to a nineteenth aspect of the present 
20 invention. In the seventeenth aspect, 

M=80 bytes. F=120 bytes. n=2 and m=3. 

A twentieth aspect of the present Invention is 

25 directed to a method of absorbing audio jitter on the side 
receiving multiplex frames when a low-bitrate-coded N- 
byte (N is a positive integer) audio frame generated at a 
rate of one frame per an F-byte (F is a positive Integer 
and N<F) transmission time is stored and multiplexed in 

30 an M-byte (M Is a positive Integer and M<F) fixed-length 
multiplex frame and when at most J-byte (J Is a positive 
integer) audio jitter is generated among the audio fields 
of the multiplex frames because FM is not an Integer, 
wherein when the time from the start of a coding 

35 (deration of an audio coder for coding audio data to the 
start of output of the first audio frame from the audio 
coder Is taken as T, and the time required from the start 
of output of the first audio frame from the audio coder to 
the start of output of the multiplex frame containing the 

40 first audio frame is taken as a, 

using as a reference the timing of the start of out- 
put of the multiplex frame containing a reference audio 
frame separated from the next audio frame at a mini- 
mum Interval in the audio frames on the transmitted 

45 multiplex frame stream, the coding operation of the 
audio coder Is started preceding the reference timing at 
least for the total time of (TW+a). 

In accordance with the twentieth aspect, it Is possi- 
ble, on the side receiving the multiplex frames, to easily 

50 absorb the audio jitter caused because the frame length 
of the multiplex frames and the cycle of transmission of 
the audio frames do not match. 

According to a twenty-first aspect of the present 
invention, in the twentieth aspect, 

55 

the time o Is set equal to or larger than a larger one 
of first and second times. 

wherein the first time is a time required from 
the start of output of the first audio frame from the 
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audio coder to the start of output of the multiplex 
frame containing the first audio frame when the 
coding operation of the audio coder is started from 
an active speech period, and 
the second time is a time required from the detec- s 
tion of the indication of silence outputted from the 
audio coder to the start of output of the multiplex 
frame containing another data frame in the field to 
be stored in an audio frame when the coding oper- 
ation of the audio coder is started from a silent 10 
period. 

According to a twenty-second aspect of the present 
invention, in the twentieth aspect, 

15 

different multiplex information are defined for n mul- 
tiplex frames containing multiplexed audio frames in 
a basic multiplex frame stream formed of m multi- 
plex frames, where n and m are integers and 
n • F=m * M , so that the reference audio frame 20 
can be identified on the basis of the multiplex infor- 
mation. 

In accordance with the twenty-second aspect, dif- 
ferent multiplex information are individually defined for n 25 
multiplex frames in which an audio frame is multiplexed, 
which allows a reference audio frame for audio jitter 
absorption to be easily identified. 

According to a twenty-third aspect of the present 
invention, in the twentieth aspect, 30 

M=80 bytes and F=120 bytes. 

According to a twenty-fourth aspect of the present 
invention, In the twenty-second aspect. 35 

M=80 bytes, F=120 bytes, n=2 and m=3. 

A twenty-fifth aspect of the present invention is 
directed to a method for multiplexing a first data frame 40 
provided with a frame number and an error detecting 
code and second to kth data frame(s) (k is an integer of 
2 or larger) in multiplex frames from a transmitting side 
to a receiving side. 

wherein the multiplex frame is further provided 4S 
with multiplex information indicating the structure of the 
multiplexing of the first data frame and the second to kth 
data frame(s), 

the transmitting side locates the frame number of 
the first data frame always in a fixed position in the mul- so 
tiplex frame, and 

when detecting presence of an error in the multi- 
plex information in the multiplex frame received, the 
receiving side reads the frame number of the first data 
frame from the fixed position in the multiplex frame and 55 
transmits a retransmission request for the read frame 
number to the transmitting side. 

In accordance with the twenty^ifth aspect, the 
frame number of tiie first data frame is always located in 



12 

a fixed position in the multiplex frame so that the receiv- 
ing side can read the frame nun*)er of the first data 
frame from the fixed position in the multiplex frame even 
if an error exists in the multiplex information in the 
received multiplex frame and can immediately transmit 
a reti^ansmission request for tiie read frame numb«^ to 
the transmitting side. 

According to a twenty-slxtin aspect of the present 
invention, in the twenty-f iftii aspect, 

a fixed-length multiplex header including at least 
tiie multiplex information and an en^or detecting 
code lor detecting an error in the multiplex informa- 
tion is located at the head of the multiplex frame, 
and 

the first data frame is always located in the first one 
of one or more data frames stored following the 
multiplex header. 

According to a twenty-severrth aspect of tiie 
present invention, in the twenty^ixth aspect, 

a header including at least the frame number and 
an error detecting code for detecting an error of tiie 
frame number is located at least at the head of tiie 
first data frame. 

According to a twenty-eightii aspect of the present 
invention, in the twenty-seventh aspect, 

when detecting presence of an error in the multqslex 
header of the multiplex frame received, the receiv- 
ing side extracts certain bytes of data following/pre- 
ceding the multiplex header as the first data frame, 
and 

when detecting absence of eaor in the header of 
tiie extracted first data frame and detecting pres- 
ence of an error In the remaining data, the receiving 
side immediately transmits a retransmission 
request for ttie first data frame to tiie ti^nsmitting 
side. 

According to a twenty-nintii aspect of tiie present 
invention, in the twenty-sixth aspect. 

the multiplex frame has a fixed length, 
tiie second data frame is always located in the last 
one of one or more data frames stored following the 
multiplex header, and 

when detecting presence of an error in tiie multiplex 
header of the multiplex frame received, the receiv- 
ing side extracts data in a range of certain bytes 
from the end of the multiplex frame as the secorxl 
data frame, 

wherein when detecting absence of an error 
in the extracted second data frame, the receiving 
side handles tiie second data frame as correctiy 
received data, and 

when detecting presence of an en-or in ttie 
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extracted second data frame, the receiving side dis- 
cards the second data frame. 

A tliirtleth aspect of the present invention is directed 
to a method for multiplexing a first data frame provided 
with a frame number and an error detecting code and 
second to kth data frame(s) (k is an integer of 2 or 
larger) in multiplex frames from a transmitting side to a 
receiving side, 

wherein the multiplex frame is further provided 
with multiplex information indicating In what structure 
the first data frame and the second to kth data frame(s) 
are multiplexed. 

wherein the transmitting side Includes as trans- 
mission modes, 

a first transmission mode In which the frame 
number of the first data frame Is always located In a 
fixed position In the multiplex frame, and 
a second transmission mode in which the frame 
number of the first data frame is located In an arbi- 
trary position in the multiplex frame, 

wherein the receiving side, when detecting 
presence of an error in the multiplex information in 
the n^jltiplex frame received in the first transmis- 
sion mode, reads the frame number of the first data 
frame from the fixed position in tiie multiplex frame 
and transmits a retransmission request for the read 
frame number to the transmitting side. 

In accordance wrth tiie thirtietii aspect, the receiv- 
ing side can read the frame number of the first data 
frame from the fixed position in the multiplex frame even 
if there Is an error in the multiplex information In the mul- 
tiplex frame transmitted in the first transmission nxxie 
and can immediately transrrdt a retransmission request 
for the read frame number to the transmitting side. 

According to a thirty-first aspect of the present 
Invention, In the thirtietii aspect, 

the tiBnsmrtting side transmits to tiie receiving side. 
In advance, carespondence data indicating the 
correspondence between tiie multiplex information 
and the multiplex structure of the multiplex frames, 
and 

the receiving side determines in which of the first 
and second transmission modes the transmitting 
side transmits the multiplex frames on the basis of 
the correspondence data received in advance. 

In accordance with the tiiirty-f Irst aspect, the receiv- 
ing side can easily determine in which of tiie first and 
second transmission modes the transnnitting side trans- 
mits the multiplex frames on the basis of the corre- 
spondence data sent from the b-ansmitting side in 
advance. 

A ttiirty-second aspect of the present invention Is 
directed to a multiplex ti'ansmisslon method in which an 
audio frame having a frame lengtii of 30 ms and being 



of 120 bytes when transmitted at 32 Kbps is low-bltrate- 
ooded, and then stored and multiplexed in a multiplex 
frame having a frame length of 20 ms and being of 80 
bytes when transmitted at 32 Kfc>ps. 

5 wherein a basic multiplex frame stream formed of 

two first multiplex frames containing tiie audio frame 
and one second multiplex frame containing none of tiie 
audio frame is repeatedly transmitted. 

In accordance witii tiie thirty-second aspect, it is 

10 possible to efficientiy multiplex audio data by almost 
intactiy using tiie structure of the ARQ frame in PIAFS. 

According to a tiitrty-tiiird aspect of the present 
invention, in the thirty-second aspect, 

15 when an audio frame to be multiplexed exists, tiie 
audio frame is always stored in the same position in 
an information field in the first nrultiplex frame, and 
a receiving side demultiplexes the audio frame from 
a multiplex frame received, absorbs jitter of tiie 
20 demultiplexed audio frame, and outputs the audio 
frame. 

In accordance with the tiilrty-third aspect, it is pos- 
sible to at)sorb audio jitter caused in multiplex transmis- 
2$ sion. 

According to a tiiirty-fourtii aspect of the present 
invention, in tiie thirty-third aspect, 

different multiplex information are defined for tiie 
30 two first multiplex frames included in the basic mul- 
t'^lex frame stream so tiiat a reference audio frame 
separated from the next audio frame at a minimum 
Interval can be identified, and 
the receiving side outputs the audio frame demulti- 
35 plexed from the received multiplex frame for each 
30 ms using tiie timing of output of the reference 
audio frame as a reference. 

In accordance witii tiie thirty-fourtii aspect, different 
40 multiplex information are defined for the two first multi- 
plex frames Included in the basic multiplex frame stream 
so that the audio frame used as a reference in absorb- 
ing the audio jitter can easily be Identified. 

According to a thirty-fiftii aspect of the present 
45 invention, in tiie thirty-second aspect, 

the first multiplex frame contains a first video frame 
which is one conplete video ARQ frame, together 
witii the audio frame, and 
50 tiie second multiplex frame contains a second video 
frame which is one complete video ARQ frame hav- 
ing a frame length equal to or larger than tiie sum of 
the frame length of the audio frame and that of the 
first video frame In the first multiplex frame. 

55 

According to a tiiirty-sixtti aspect of the present 
invention. In tiie thirty-fiftii aspect, 

when tiie audio frame is not transmitted In a silent 
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period, the first multiplex frame is used as a third 
multiplex frame which forms a third video frame 
which is one complete video ARQ frame having a 
frame length equal to the sum of the frame lengths 
of the audio frame and the first video frame, and 
when the third video frame could not be normally 
received, a retransmission thereof Is performed in 
the second or third multiplex frame. 

In accordance with the thirty-sixth aspect, when 
video data transmitted in the third video frame is 
errored, the video data can be retransmitted in either of 
the second and third multiplex frames. 

According to a thirty-seventh aspect of the present 
invention, in the thirty-fifth aspect, 

a fixed-length multiplex header at least including 
multiplex information Indicating the internal multi- 
plex structure and an error detecting code for 
detecting eror of the multiplex information is 
located at the head of each of the first and second 
multiplex frames, and 

in each of the first and second multiplex frames, an 
ARQ header for each of the first and second video 
frames is stored immediately after the multiplex 
header or at the end of each of the first and second 
multiplex frames. 

wherein when an error is detected in the 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame Including the 
ARQ header is read and retransmission control 
thereof is performed. 

In accordance with the thirty-seventh aspect, the 
ARQ headers for the first and second video frames are 
stored in a fixed position in the multiplex headers. This 
allows the receiving side to read the ARQ headers for 
the first and second video frames from the fixed position 
in the multiplex frames even if there is an error in the 
muHplex information in the received multiplex frame 
and to perform the retransmission control at once. 

According to a tiiirty-eighth aspect of the present 
invention, in the thirty-fifth aspect, 

a fixed-length multiplex header at least including 
multiplex information indicating the internal multi- 
plex structure and an error detecting code for 
detecting error of the multiplex information is 
located at the head and at tiie end of each of the 
first and second multiplex frames, and 
in each of the first and second multiplex frames, an 
ARQ header for each of the first and second video 
frames is stored immediately after the multiplex 
header located at the head of each of the first and 
second multiplex frames or immediately before tiie 
multiplex header located at the end of each of tiie 
first and second multiplex frames, 

wherein when an error is detected in tiie 
multiplex header and tiie internal multiplex struc- 



ture is not known, the video frame including tiie 
ARQ header is read and retransmission control 
tiiereof is performed. 

5 In accordance with the thirty-eighth aspect, tiie 
ARQ headers for the first and second video frames are 
each stored in a fixed position in the multiplex headers. 
This allows the receiving side to read the ARQ headers 
for tiie first and second video frames from the fixed posl- 

10 tion in the multiplex frames even if there is an error in the 
multiplex information in a received multiplex frame and 
can perform the retransmission control at once. 

A thirty-ninth aspect of the present invention is 
directed to a multiplex transmission method in which an 

15 audio frame having a frame length of 30 ms and being 
of 120 bytes when transmitted at 32 Kbps is low-bitrate- 
coded, and then stored and multiplexed in a multiplex 
frame having a frame length of 15 ms and being of 60 
bytes when transmitted at 32 Kbps, 

20 wherein a first multiplex frame containing tiie 

audio frame and a second multiplex frame containing 
none of the audio frame are alternately and repeatedly 
transmitted. 

According to a fortieth aspect of tiie present inven- 
ts tion, in tiie thirty-ninth aspect, 

tiie first multiplex frame contains a first video frame 
which is one complete video ARQ frame, together 
witii tiie audio frame, and 
30 the second multiplex frame contains a second video 
frame which is one complete video ARQ frame hav- 
ing a frame length equal to or larger than the sum of 
the frame length of the audio frame and that of the 
first video frame in tiie first multiplex frame. 

35 

According to a forty-first aspect of tiie present 
invention, in the fortietii aspect 

when tiie audb frame is not tiansmrtted in a silent 
40 period, tiie first multiplex frame is used as a third 
multiplex frame which fomis a tiiird video frame 
which is one complete video ARQ frame having a 
frame length equal to the sum of the frame length of 
tiie audio frame and that of the first video frame, 
45 and 

when the third video frame can not be normally 
received, a retransmission thereof is performed in 
the second or third multiplex frame. 

so In accordance with the forty-first aspect when 
video data transmitted In the tiiird video frame is 
errored, the video data can be retransmitted In any of 
the secorxj and tiiird multiplex frames. 

According to a forty-second aspect of the present 

55 invention, in the fortietii aspect, 

a fixed-length multiplex header at least including 
multiplex infomiation indicating tiie internal multi- 
plex structure and an error detecting code for 
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detecting error of the multiplex Information is 
located at the head of each of the first and second 
multiplex frames, and 

in each of the first and second multiplex frames, an 
ARQ header for each of the first and second video 
frames is stored immediately after the multiplex 
header or at the end of each of the first and second 
multiplex frames, 

wherein when an error is detected in the 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame Including the 
ARQ header is read and retransmission control 
thereof is performed. 

In accordance with the forty-second aspect, the 
ARQ headers for the first and second video frames are 
stored in a fixed position in the multiplex headers. This 
allows the receiving side to read the ARQ headers for 
the first and second video frames from the fixed position 
in the multiplex frames even if there is an error in the 
multiplex information in the received multiplex frame 
and to perform the retransmission control at once. 

According to a forty-third aspect of the present 
invention, in the fortieth aspect, 

a fixed-length multiplex header at least including 
multiplex information indicating the internal multi- 
plex structure and an en^or detecting code for 
detecting en-or of the multiplex Information is 
located at the he^ and at the end of each of the 
first and second multiplex frames, 
in each of the first and second multiplex frames, an 
ARQ header for each of the first and second video 
frames is stored immediately after the multiplex 
header located at the head of each of the first and 
second multiplex frames or immediately before the 
multiplex header located at the end of each of the 
first and second multiplex frames, 

wherein when an error is detected in the 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame including the 
ARQ header Is read and retransmission control 
thereof is performed. 

In accordance with the forty-third aspect, the ARQ 
headers for the first and second video frames are stored 
in a fixed position in the multiplex headers. This allows 
the receiving side to read the ARQ headers for the first 
and second video frames from the fixed position in the 
multiplex frames even if there is an error in the multiplex 
information In the received multiplex frame and to per- 
form the retransmission control at once. 

A forty-fourth aspect of the present invention is 
directed to a system for multiplexing variable-length 
data employing retransmission to wNch error correction 
by retransmission control is applied and fixed-length 
data not employing retransmission to which error cor- 
rection by retransmission control is not applied in multi- 
plex frames from a transmitting device to a receiving 



device. 

wherein the transmitting device provides the mul- 
tiplex frame with a fixed length and increases/deaeases 
the frame length of a data link frame for storing the var- 

5 iable-length data employing retransmission depending 
on the length of the multiplexed fixed-length data not 
employing retransmission to transmit the data link frame 
at a timing always in synchronization with the transmis- 
sion stream of the multiplex frames. 

10 In accordance with the forty-fourth aspect, multiplex 
frames for storing the fixed-length data not employing 
retransmission and the variable-length data employing 
retransmission have a fixed length and data link frames 
for storing the variable-length data employing retrans- 

15 mission have a frame length changed depending on the 
length of the fixed-length data not employing retrans- 
mission to be multiplexed so that the data link frames 
can be transmitted always In synchronization with the 
transmission stream of the multiplex frames. Then the 

20 transmission timing of the data link frames can be fixed 
and the boundaries between the data link frames are 
not lost in the event of a transnmssion error, providing 
en^or robustness. That is to say, unlike H.223 which 
acquires synchronization with flags, bits are not shifted 

25 due to "0" insertion, and even rf the slot synchronization 
is lost due to a transmission error, it Is possible to see 
the next header at once, providing easy resynchroniza- 
tion. Moreover, tt is possible to effectively utilize silent 
periods of the fixed-length data not employing retrans- 

30 mission to Inaease the transmission efficiency of the 
video data, or the variable-length data employing 
retransmission. 

According to a forty-fifth aspect of the present 
invention, in the forty-fourth aspect, 

35 

the multiplex frame includes a fixed-length field, 
the fixed-length field being divkled into first and 
second variable-length slots, 
the first variable-length slot having a predetermined 

40 fixed length when fixed-length data not employing 
retransmission to be multiplexed exists and having 
a length of zero when no fixed-length data not 
employing retransmission to be multiplexed exists, 
the second variable-length slot having a relatively 

45 short, first length when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a relatively long, second length when no 
fixed-length data not employing retransmission to 
be multiplexed exists, 

50 wherein the transmitting device transmits the 

fixed-length data not employing retransmission in 
the first variable-length slot and transmits the varia- 
ble-length data employing retransmission in the 
second variattle-tength slot 

55 

According to a forty-sixth aspect of the present 
invention, in the forty-fifth aspect, 

the transmitting device has a first data link process- 
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Ing block performing transmission control of the var- 
iable-length data employing retransmission only in 
the form of a fixed-length ARQ frame, 
the first data link processing block generates a 
fixed-length ARQ frame in which the data length of s 
significant data in the variable-length data employ- 
ing retransmission Is increased/decreased dep^d- 
ing on whether the fixed-length data not employing 
retransmission to be multiplexed exists and dummy 
bits are inserted in the remaining part and 10 
the transmitting device removes the dummy bits 
from the ARQ frame to convert the ARQ frame into 
a data link frame of variable-length data employing 
retransmission having a frame length equal to the 
length of the second variable-length slot, and is 
multiplexes the data link frame of the variable- 
tength data employing retransmission and the data 
frame of the fixed-length data not employing 
retransmission to generate the multiplex frame to 
be transmitted to the receiving device. 20 

In accordance with the forty-sixth aspect, the frame 
timing of the data link frames of the variable-length data 
employing retransmission can have the same cycle and 
the same rate as the frame timing of PIAFS for the PHS 26 
data communication. 

According to a forty-seventh aspect of the present 
invention, in the forty-sixth aspect, 

the receiving device has a second data link 30 
processing block performing transmission control of 
the variable-length data employing retransmission 
only in the form of a fixed-length ARQ frame, 
the receiving device demultiplexes a received multi- 
plex frame into the data link frame of the variable- 35 
lengtii data employing retransmission and the data 
frame of the fixed-length data not employing 
retransmission, 

inserts dummy bits into the demultiplexed data link 
frame of the variable-length data employing 40 
retransmission depending on whether the received 
multiplex frame includes the fixed-lengtii data not 
employing retransmission to convert the data link 
frame of the variable-length data employing 
retransmission Into tiie fixed-length ARQ frame. 45 
and 

gives tiie ARQ frame obtained by the conversion to 
the second data link processing block. 

In accordance with the forty-seventh aspect, it is so 
possible to easily convert received data into an ARQ 
frame of PIAFS for the PHS data communication, which 
allows a processing block for performing usual PHS 
data communication to process the data. 

According to a forty-eighth aspect of the present 55 
invention, in the forty-seventh aspect, 

the frame length of tiie multiplex frame is equal to 
that of the ARQ frame. 



A forty-nintii aspect of the present invention is 
directed to a metiiod for performing multiplex transmis- 
sion between a transmitting side and a receiving side by 
using multiplex frames on a conrposite data transmis- 
sion network in which a first network performing syn- 
chronous communication and a second network 
performing asynchronous communication are con- 
nected to each other, 

wherein multiplex frames containing synchroni- 
zation flags at tiie heads are always transmitted from 
tiie transmitting side to tiie receiving side while tiie com- 
munication is continuing, and 

when a communication is made within tiie first 
network, frame synchronization established prior to tiie 
data communication is kept by using the multiplex 
frames having a fixed frame lengtii. without using tiie 
synchronization flags. 

In accordance witii tiie forty-ninth aspect, it is pos- 
sible to realize the synchronous communication deter- 
mined by PIAFS for PHS data communication while 
following tiie multiplex transmission method determined 
in H.223. 

A fiftieth aspect of the present invention is directed 
to a method for performing multiplex transmission 
between a transmitting side and a receiving side by 
using multiplex frames In a composite data transmission 
network in which a first network performing synchro- 
nous communication and a second network performing 
asynchronous communication are connected to each 
other, 

wherein when the second network is interposed 
between tiie transmitting side and the receiving side, 
frame synchronization is acquired between the trans- 
mitting side and the receiving side by transm'rtting multi- 
plex frames containing synchronization flags at tiie 
heads, and 

when making a communication wittiin tiie first 
network, multiplex frames containing no synchroniza- 
tion flags are transmitted and frame synchronization 
established prior to the data communication is kapi by 
using the multiplex frames having a fixed frame length. 

In accordance with tiie fiftietii aspect, It Is possible 
to realize tiie synchronous communication determined 
by PIAFS for PHS data communication while following 
tiie multiplex transmission method determined In H.223. 
Furthermore, when a communication is made within the 
first network, multiplex frames containing no synchroni- 
zation flags are transmitted, Inproving the transmission 
efficiency. 

A fifty-first aspect of the present invention is 
directed to a method for multiplexing data employing 
retransmission to which en'or correction by retransmis- 
sion control is applied and data not employing retrans- 
mission to which error correction by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path In which a first trans- 
mission path witii a relatively high bit rate and a second 
transmission patii with a relatively low bit rate are con- 
nected to each other. 
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wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 
sion path, the data not employing retransmission in 
which a transmission en^or has occun-ed on the first 
transmission path is discarded at the connection from 
the first transmission path to the second transmission 
path. 

In accordance with the fifty-first aspect, data not 
enploying retransmission in which a transmission error 
has occurred on the first transmission path is discarded 
at the connection from the first transmission path to the 
second transmission path. This prevents unwanted data 
from being outputted onto the second transmission 
path, which prevents reduction of the bit rate on the sec- 
ond transmission path. 

A fifty-second aspect of the present tnverrtion is 
directed to a method for multiplexing data employing 
retransmission to which error connection by retransmis- 
sion control is applied and data not employing retrans- 
mission to which en^or correction by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path in which a first trans- 
mission path with a relatively high bit rate and a second 
transmission path with a relatively low bit rate are con- 
nected to each other, 

wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 
sion path, 

the data employing retransmission is provided 
with a block-coded error correcting code and transmit- 
ted on the first transmission path, 

at the connection from the first transmission path 
to the second transmission path, the data employing 
retransmission is subjected to error correction with the 
error connecting code and then outputted onto the sec- 
ond transmission path with the error correcting code 
removed therefrom, and 

retransmission control thereof is performed 
between a transmitting terminal and a receiving termi- 
nal. 

In accordance with the fifty-second aspect, at the 
connection from the first transmission path to the sec- 
ond transmission path, error confection only is applied to 
the data employing retransmission and retransmission 
control is not applied to the data employing retransmis- 
sion. Then the error correcting code is removed and the 
data err^loying retransmission is outputted onto the 
second transmission path. This lessens the amount of 
data outputted onto the second transmission path, 
which prevents reduction of the bit rate on the second 
transmission path. 

A fifty-third aspect of the present invention is 
directed to a method for multiplexing data employing 
retransmission to which error correction by retransmis- 
sion control is applied arxj data not employing retrans- 
mission to which ennor correction by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path in which a first trans- 
mission path with a relatively high bit rate and a second 



transmission path with a relatively low bit rate are con- 
nected to each other, 

wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 
5 sion path, 

the data employing retransmission is provided 
with a block-coded error correcting code and transmit- 
ted on the first transmission path, 

at the connection from the first transmission path 
10 to the second transmission path, the en-or correcting 
code is just removed, without error correction with the 
en^or connecting code, and then the data employing 
retransmission is outputted onto the second transmis- 
sion path, and 

15 retransmission control thereof is performed 

between a transmitting terminal and a receiving termi- 
nal. 

In accordance with the fifty-third aspect, at the con- 
nection from the first transmission path to the second 

20 transmission path, retransmission control to the data 
employing retransmission is not performed and the enror 
correcting code is just removed, and the data employing 
retransmission is then outputted onto the second trans- 
mission path. This lessens the amount of data outputted 

25 onto the second transmission path, which prevents 
reduction of the bit rate on the second transmission 
path. 

A frfty-fburth aspect of the present invention is 
directed to a method for multiplexing data employing 

30 retransmission to which error connection by retransmis- 
sion control is applied and data not employing retrans- 
mission to which ennor correction by retransmission 
control is not applied by using multiplex frames on a 
composite data trarremission path in which a first trans- 

35 mission path with a relatively high bit rate and a second 
transmission path with a relatively low bit rate are con- 
nected to each other, 

wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 

40 sion path, 

the data employing retransmission is corrvolu- 
tional-coded with a certain code rate Rc and transmitted 
on the first transmission path, 

at the connection from the first transmission path 

45 to the second transmission path, the convolutional cod- 
ing is decoded and then the data employing retransmis- 
sion is outputted onto the second transmission path, 
and 

its retransmission control is performed between 
50 a transmitting terminal and a receiving terminal . 

In accordance with the fifty-tburth aspect, at the 
connection from the first transmission path to the sec- 
ond transmission path, the convolutional-coded data 
enploying retransmission is just decoded without 
55 retransmission control, which is outputted onto the sec- 
ond transmission path without re-coded. This lessens 
the amount of data outputted onto the second tiansmis- 
sion path, which prevents reduction of the bit rate of the 
second transmission path. 
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According to a fifty-fifth aspect of tlie present inven- 
tion, in the fifty-fourth aspect, 

when Rv expresses a value obtained from the bit 
rate Vf of the first transmission path and the bit rate 
Vs of the second transmission path, 
{Rv=(Vf- Vs)/Vf}, 

Lm expresses the frame length of the multiplex 
frame, 

Ld expr^ses the data length of the data employing 
retransmission subjected to the convolutional cod- 
ing, and 

Lo expresses the length of overhead newly added 
at the time of transmission on the second transmis- 
sion path, then 

the certain code rate Rc is calculated as 

Rc=Ld/(Lm • Rv-Lo+Ld). 

A fifty-sixth aspect of the present invention is 
directed to a method for multiplexing data employing 
retransmission to which error correction by retransmis- 
sion control is applied and data not employing retrans- 
mission to which error correction by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path in which a first trans- 
mission path with a relatively high bit rate and a second 
transmission path with a relatively low bit rate are con- 
nected to each other, 

wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 
sion path, 

the data not employing retransmission is convo- 
lutional-ooded with a certain code rate Rc and transmit- 
ted on the first transmission path, and 

at the connection from the first transmission path 
to the second transmission path, the convolutional cod- 
ing is decoded and then the data not employing retrans- 
mission is oulputted onto the second transmission path. 

In accordance with the fifty-sixth aspect, at the con- 
nection from the first transmission path to the second 
transmission path, the oonvoluttonal-coded data not 
employing retransmission is just decoded and is output- 
ted onto the second transmission path without re- 
coded. This lessens the amount of data outputted onto 
the second transmission path, which prevents reduction 
of the bit rate of the second transmission path. 

According to a fifty-seventh aspect of the present 
invention, in the fifty-sixth aspect, 

when Rv expresses a value obtained from the bit 
rate Vf of the first transmission path and the bit rate 
Vs of the second transmission path, 
{Rv=(Vf- Vs)/Vf}, 

Lm expresses the frame length of the multiplex 
frame, 

Lxl represses the data length of the data not 
employing retransmission subjected to the convolu- 
tional coding, and 



Lo expresses the length of overhead newly added 
at the time of transmission on the second transmis- 
sion path, then 

the certain code rate Rc is calculated as 

5 

Rc=Ld/{Lm • Rv-Lo+Ld). 

A fifty-eighth aspect of the present invention is 
directed to a method for multiplexing data employing 

10 retransmission to which en^or correction by retransmis- 
sion control is applied and data not employing retrans- 
mission to which error corection by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path in which a first trans- 

15 mission path with a relatively high bit rate and a secorxi 
transmission path with a relatively low bit rate are con- 
nected to each other. 

wherein when the multiplex frame is transmitted 
from the first transmission path to the second transmis- 

20 sion path. 

the data not employing retransmission is pro- 
vided with a block-coded error correcting code and 
transmitted on the first transmission path, and 

at the connection from the first transmission path 

25 to the second transmission path, the data not employing 
retransmission is subjected to error correction with the 
error correcting code and then oulputted onto the sec- 
ond transmission path with the error correcting code 
removed therefrom. 

30 In accordance with the fifty-eighth aspect, at the 
connection from the first transmission path to the sec- 
ond transmission path, the data not employing retrans- 
mission is just subjected to error correction and the 
error correcting code is removed, and then the data not 

35 employing retransmission is outputted onto the second 
transmission path. This lessens the amount of data out- 
putted onto the second transmission path, which pre- 
vents reduction of the bit rate of the second 
transmission path. 

40 A fifty-ninth aspect of the present invention is 
directed to a method for multiplexing data employing 
retransmission to which en^or correction by retransmis- 
sion control is applied and data not employing retrans- 
mission to which error correction by retransmission 

45 control is not applied by using multiplex frames on a 
composite data transmission path in which a first trans- 
mission path with a relatively high bit rate and a second 
transmission path with a relatively low bit rate are con- 
nected to each other, 

50 wherein when the multiplex frame is transmitted 

from the first transmission path to the second transmis- 
sion path, 

the data not employing retransmission is pro- 
vided with a tslock-ooded error correcting code and 
55 transmitted on the first transmission path, and 

at the connection from the first transmission path 
to the second transmission path, the error correcting 
code is just removed, without error correction with the 
error correcting code, and then the data not employing 



13 



25 



EP0 806 873 A2 



26 



retransmission is outputted onto the second transmis- 
sion path. 

In acxx}rdance with the fifty-ninth aspect, at the con- 
nection from the first transmission path to the second 
transmission path, the enror correcting code is just s 
removed and then the data not employing retransmis- 
sion is outputted onto the second transmission path. 
This lessens the amount of data outputted onto the sec- 
ond transmission path, which prevents reduction of the 
bit rate on the second transmission path. 10 

A sixtieth aspect of the present invention is directed 
to a method for multiplexing data employing retransmis- 
sion to which error correction by retransmission control 
is applied and data not employing retransmission to 
which error correction by retransmission control is not is 
applied by using multiplex frames on a composite data 
transmission path in which a first transmission path with 
a relatively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each other. 

wherein when the multiplex frame is transmitted 20 
from the first transmission path to the second transmis- 
sion path. 

the data employing retransmission is convolu- 
tional-coded with a certain code rate Rc and transmitted 
on the first transmission path, and 2s 

at the connection from the first transmission path 
to the second transmission path, the convolutional cod- 
ing is decoded and then the data enploying retransmis- 
sion is outputted onto the second transmission path, 
and 30 

its retransmission control is performed between 
a transmitting terminal and a receiving terminal, and 

the data not employing retransmission is oonvo- 
lutional-coded with a certain code rate Rc arxJ transmit- 
ted on the first transmission path, arxj 35 

at the connection from the first transmission path 
to the second transmission path, the convolutional cod- 
ing is decoded and then the data not employing retrans- 
mission is outputted onto the second transmission path. 

In accordance with the sixtieth aspect, at the con- 40 
nection from the first transmission path to the second 
transmission path, the oonvolutional-coded data 
employing retransmission is decoded without being 
subjected to retransmission control, and then the data 
employing retransmission is outputted onto the second 45 
transmission path without being re-coded. This lessens 
the amount of the data employing retransmission out- 
putted onto the second transmission path. Also, at the 
connection from the first transmission path to the sec- 
ond transmission path, the convolutionat-coded data not so 
employing retransmission is decoded and outputted 
onto the second transmission path without being coded 
again. This lessens the amount of the data not employ- 
ing retransmission outputted onto the second transmis- 
sion path. 55 

According to a sixty-first aspect of the present 
invention, in the sixtieth aspect, 

when Rv expresses a value obtained from the bit 



rate Vf of the first transmission path and the bit rate 
Vs of the second transmission path, 
{Rv=:(Vf- Vs)A/f}, 

Lm expresses the frame length of the nnultiplex 
frame, 

Ld expresses the total of the data lengths of the 
data employing retransmission and the data not 
employing retransmission subjected to the convolu- 
tional coding, and 

Lo expresses the length of overhead newly added 
at the time of transmission on the second transmis- 
sion path, then 

the certain code rate Rc is calculated as 
Rc=ljd/(Ljn • Rv-Lo+Ld). 

A sixty-second aspect of the present invention is 
directed to a method for multiplexing and transmitting 
variable-length data employing retransmission to which 
en-or correction by retransmission control is applied and 
fixed-length data not employing retransmission to which 
en-or correction by retransmission control is not applied 
in multiplex frames. 

wherein the multiplex frannes have a fixed length 
and a data link frame for storing the variable-length data 
employing retransmission has a frame length 
increased/decreased depending on the length of the 
fixed-length data not employing retransmission to be 
multiplexed so that the data link frame is transmitted at 
timing kept in synchronization with the transmission 
stream of the multiplex frames. 

In accordance with the sixty-second aspect, the 
multiplex frames for storing the fixed-length data not 
enrploying retransmission and the variable-length data 
enrploying retransmission have a fixed length and a 
data link frame for storing the variable-length data 
employing retransmission has a frame length changed 
depending on the ler^gth of the fixed-length data not 
employing retransmission to be multiplexed so that the 
data link frames can be transmitted at timing synchro- 
nized with the transmission stream of the multiplex 
frames. Then the transm'rtting timing of the data link 
frames can be fixed and the bourxlaries between the 
data link frames are not lost in the event of a transmis- 
sion enror, providing error robustness. That is to say, 
unlike H.223 which acquires synchronization with flags, 
bHs are not shifted due to "0" insertion, and even if the 
slot synchronization is lost due to a transmission en-or, it 
is possible to see the next header at once, providing 
easy resynchronizatlon. Moreover, it is possible to effec- 
tively utilize silent periods of the fixed-length data not 
employing retransmission to increase the transmission 
efficiency of the video data, or the variabile-length data 
enrploying retransmission. 

A sixty-third aspect of the present invention is 
directed to a storage medium for storing software pro- 
gram to be executed in order to multiplex and transmit 
variable-length data employing retransmission to which 
en-or correction by retransmission control is applied and 
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fixed-length data not emplying retransmission to which 
error correction by retransmission control is not applied 
in multiplex frames. 

wherein said software program fix a length of 
said multiplex frame length and increase/decrease the 5 
length of data link frame length for storing said variat}le- 
length data employing retransmission depending on the 
length of said fixed-length data not emplying retrans- 
mission to be multiplexed so that the data link frame is 
transmitted at timing kept in synchronization, for each w 
said multiplex frame, with the transmission stream of 
said multiplex frames. 

According to a sixty-forth aspect of the present 
invention, in the sixty-third aspect, when fixed-length 
data not employing retransmission to be multiplexed is 
exists, a basic multiplex frame stream Including at least 
one first multiplex frame in which said fixed-length data 
not employing retransmission is multiplexed and at least 
one second multiplex frame in which said fixed-length 
data not employing retransmision is not multiplexed is 20 
repeatedly transmitted. 

A sixty-fifth aspect of the present invention is 
directed to a storage medium for storing software pro- 
gram to be excuted in order to absorb the audio jitter on 
the side receiving multiplex frames when a low-bitrate- ss 
coded N-byte (N is a positive integer) audio frame gen- 
erated at a rate of one frame per an F-byte (F is a posi- 
tive integer and N<F) transmission time is stored and 
muftiplexed In an M-byte (M is a positive integer and 
M<F) fixed-length nnultiplex frame and when at most J- 30 
byte (J is a positive integer) audio jitter is generated 
among the audio fields of the multiplex frames becai^e 
F/M is not an integer, 

wherein said software program comprises 

35 

a first program step for identif ing a reference audio 
frame separated from the next audio frame at a 
minimum interval among tiie audio frames on a 
stream of tiie transmitted multiplex frame, and 
a second program step for oulputting the following 40 
audio frames at the rate of one frame per tiie F-byte 
transmission time using tiie timing at which said ref- 
erence audio frame is outputted as a reference. 

wherein different multiplex information are 
defined tor n multiplex frames containing multi- 45 
plexed audio frames in a basic multiplex frame 
stream fomfied of m multiplex frames, where n and 
m are integers and n • F=m • M , and 
said reference audio frame is identified on the basis 
of said multiplex information at said first program so 
step. 

A sixty-sixth aspect of the present invention is 
derected to a storage medium for storing software pro- 
gram to be excuted In order to absorb tiie audio jitter on ss 
the side transmitting multiplex frames when a low- 
bitrate-coded N-byte (N is a positive integer) audio 
frame generated at a rate of one frame per an F-byte (F 
is a positive integer and N<F) transmission time is 



stored and multiplexed in an M-byte (M is a positive inte- 
ger and M<F) fixed-lengtii multiplex frame and when at 
most J-byte (J is a positive integer) audio jitter is gener- 
ated among the audio fields of the multiplex frames 
because F/M is not an integer, 

wherein when the time from tiie start of a coding 
operation of an audio coder for coding audio data to the 
start of output of the first audio frame from the audio 
coder is taken as T, and tiie time required from the start 
of output of the first audio frame from said audio coder 
to the start of output of the multiplex frame containing 
said first audio frame is taken as a. 

using as a reference the timing of tiie start of out- 
put of tiie multiplex frame storing a reference audio 
frame separated from tiie next audio frame at a mini- 
mum interval in the audio frames on the transmitted 
multiplex frame stream, the coding operation of said 
audio coder is started preceding the reference timing at 
least for the total time of (T-Ki-f a), and 

wherein different multiplex information are 
d^ined for n multiplex frames containing multiplexed 
audio frames in a basic multiplex frame stream fomied 
of m multiplex frames, where n and m are integers and 
n • F=m • M , and 

said reference audio frame is identified on tiie 
basis of said multiplex information by said software pro- 
gram. 

A sixty-seventii aspect of tiie present invention is 
directed to a storage medium for storing software pro- 
gram to be executed in order to store in multiplex 
frames, multiplex and tranmit a first data frame provided 
with a frame number and an error detecting code and 
second to kth data frame(s) (k is an integer of 2 or 
larger), 

wherein said multiplex frame Is further provided 
with multiplex information indicating the structure of tiie 
multiplexing of said first data frame and said second to 
kth data frame(s). and 

wherein said software program has 

said transmitting side locate the frame number of 
said first data frame always in a fixed position in said 
multiplex frame and 

when detecting presence of an error in said mul- 
tiplex information in the multiplex frame received, said 
receiving side read the frame number of said first data 
frame from the fixed position in the nniltiplex frame and 
transmit a retransmission request for tiie read frame 
number to said transmitting side. 

These and otiier objects, features, aspects and 
advantages of tiie present invention will become more 
apparent from the following detailed description of tiie 
present invention when taken in conjunction witii tiie 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Rg.1 is a diagram showing the structure of a fixed- 
lengtti multiplex frame used in a first embodiment of tiie 
present invention. 
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Fig.2 is a diagram showing the structure of a fixed- 
length nuiltiptex frame used in a second embodiment of 
the present invention. 

Fig.3 is a diagram showing the multiplexing in the 
first or second embodiment, wherein audio data coded s 
by G.723 is multiplexed with video data in the multiplex 
frames having a frame length set to 120 bytes and 
transmitted through a transmission path at 32 Kbps of 
PHS which is a simplified mobile telephone. 

Fig.4 is a diagram showing video slots of different 
sizes in a speech period and in a silent period in the 
multiplex frame shown in Fig.3. 

Fig.5 is a diagram showing the structure of a f ixed- 
fength multiplex frame used in a third embodiment of the 
present invention. 

Fig.6 is a diagram showing the structure of a fixed- 
length multiplex frame used In a fourth embodiment of 
the present invention. 

Fig.7 is a diagram showing the structure of a fixed- 
length multiplex frame used in a fifth embodiment of the 
present invention. 

Fig.8 is a diagram showing the structure of a fixed- 
length multiplex frame in the fourth embodiment, in 
which the frame length of each fixed-length multiplex 
frame is changed from 60 bytes to 80 bytes. 

Fig.9 is a diagram showing the structure of a fixed- 
length multiplex frame in the fifth embodiment, in which 
the frame length of each fixed-length multiplex frame is 
changed from 60 bytes to 80 bytes. 

Fig. 10 is a layer diagram showing the structure of 
multiplex transmission devices in a sixth embodiment of 
the present invention. 

Fig. 11 is a diagram showing a first example of 
structure of a fixed-length multiplex frame used in the 
multiplex transmission device of Fig. 10. 

Fig. 12 is a diagram showing a second example of 
structure of a fixed-length multiplex frame used in the 
multiplex transmission device of Fig.10. 

Fig. 13 Is a diagram showing a first structure and tiie 
data reading timing of a fixed-length multiplex frame 
used in seventii and eighth embodiments of the present 
invention. 

Fig. 14 is a diagram showing a first example of a 
multiplex code table desaibing the relation between the 
multiplex structure and the multiplex information of the 
fixed-length multiplex frames. 

Fig. 15 is a diagram showing a second example of a 
multiplex code table desaibing the relation between the 
multiplex structure and the multiplex information of the 
fixed-length multiplex frames. 

Fig. 16 is a diagram showing a second structure and 
the data reading timing of a fixed-length multiplex frame 
used in the seventh embodiment of the present inven- 
tion. 

Fig. 17 is a diagram showing a third structure and 
the data reading timing of a fixed-length multiplex frame 
used in the seventh embodiment of the present inven- 
tion. 

Fig. 18 is a diagram lilustiBting tiie reason why data 



is not read immediately after arrival of audio data in 
Fig.17. 

Fig. 19 is a diagram showing a third exanrple of a 
multiplex code table describing the relation between the 
multiplex structure and the multiplex information of tiie 
fixed-length multiplex frames. 

Fig.20 is a flow chart showing processing on tiie 
receiving terminal side in a ninth embodiment of tiie 
present invention. 

Fig.2l is a diagram showing a fourth example of a 
multiplex code table describing the relation between tiie 
multiplex structure and the multiplex information of tiie 
fixed-length multiplex frames. 

ng.22 is a diagram showing a second structure and 
the data reading timing of a fixed-lengtii multiplex frame 
used in the eightii embodiment of the present invention. 

Rg.23 is a diagram showing a third structure and 
the data reading timing of a fixed-length multiplex frame 
used in the eightii embodiment of tiie present invention. 

Rg.24 is a timing chart showing a first example of 
operation on tiie ti^ansmitting side in a tenth embodi- 
ment of the present Invention. 

Rg.25 is a timing chart showing a second example 
of operation on the transmitting side in tiie tenth embod- 
iment of the present invention. 

Fig.26 is a timing chart showing a tiiird example of 
operation on the transmitting side in the tenth emtxxJi- 
ment of the present invention. 

Rg.27 is a diagram showing a multiplex structure of 
multiplex frames used in an eleventh emt>odiment of tiie 
present Invention. 

Rg.2d is a diagram showing a multiplex structure of 
multiplex frames used in the eleventh embodiment of 
the present invention. 

Rg.29 is a flow chart showing processing per- 
formed by a multiplex layer on the receiving side in tiie 
eleventii embodiment of the present invention. 

Rg.30 is a flow chart showing processing per- 
formed by a video en^or control processing layer in tiie 
eleventh embodiment of the present invention. 

Fig.31 Is a timing chart showing operation with 
retransmission control performed in tiie eleventh 
embodiment of the present invention. 

Rg.32 Is a flow chart showing processing per- 
formed by a multiplex layer on tiie receiving side in a 
twelfth embodiment of tiie present invention. 

Fig.33 is a flow chart showing processing per- 
formed by a video error control processing layer in tiie 
twelfth embodiment of tiie present invention. 

Rg.34 is a flow chart showing processing per- 
formed by a multiplex layer on the receiving side in tiie 
twelfth embodiment of tiie present invention. 

Fig.35 is a diagram for describing a multiplex trans- 
mission method according to a thirteenth embodiment 
of the present invention. 

Fig.36 Is a diagram illustrating a multiplex transmis- 
sion method with each multiplex frame having a length 
of 60 bytes in the thirteenth embodiment of tiie present 
invention. 
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Fjg.37 is a diagram showing tlie structure of a com- 
munication system to wliich a fourteenth embodiment of 
the present invention is applied. 

Fig.38 Is a diagram illustrating a multiplexing 
method at a terminal connected to the PHS network in s 
the fourteenth embodiment of the present Invention. 

Fig.39 is a diagram for illustrating a method of con- 
verting the rate at the network connection in the four- 
teenth embodiment of the present invention. 

Fig.40 is a block diagram showing the structure of 
the network connection in a fifteenth embodiment of the 
present invention. 

Fig.41 is a diagram showing a structure of a com- 
munication system to which sixteenth and seventeenth 
embodiments of the present invention are applied. 

Fig.42 is a diagram showing the structure of a mul- 
tiplex frame inputted to the interconnecting device and 
the structure of the multiplex frame outputted from the 
interconnecting device in the sixteenth embodiment of 
the present invention. 

Fig.43 is a diagram showing the structure of a mul- 
tiplex frame transmitted by a terminal connected to the 
PHS network in the seventeenth embodiment of the 
present invention. 

Fig.44 is a diagram showing a common layer struc- 
ture of a conventional multiplex transmission device 
which adopts the multiplex transmission method deter- 
mined by H.223. 

Fig. 45 is a diagram showing the multiplex frame for- 
mat in the multiplex layer in H.223. 

Fig.46 is a diagram showing an example of a multi- 
plex frame format in the multiplex layer in the case 
where audio data, computer data and video data are 
multiplexed. 

Fig.47 is a diagram showing the format of the ARQ 
frame for error correction by retransmission control of 
video. 

Fig.46 is a diagram showing the frame structure 
defined in PIAFS. 

Fig. 49 is a diagram for describing "Mixing Transfer 
System for HDLC Variable Length Packet and None 
HDLC Fixed Length Packet" disdosed in Japanese Pat- 
ent Publication No.8-13057. 

Rg.50 is a block diagram showing a structure of a 
conventional jitter correcting circuit. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

(First Prefen^ed Embodiment) 

An object of a first embodiment is to allow both of 
multiplex frames and video frames to easily acquire 
frame synchronization to provide a high tolerance for 
errors, when multiplexing fixed-length data not employ- 
ing retransmission (audio data coded according to 
G.723 herein) ard variable-length data employing 
retransmission (video data coded by H.263 herein). 

G.723 is a method for coding audio data to a rate of 



5.3 Kbps, whose basic unit is formed of a 20-byte fixed- 
length frame. The fixed-length frames continue in a 
speech period in which audio is generated. Hence, 
when multiplexed on a transmission path with a higher 
rate, e.g., on a transmission path of 32 Kbps of the sim- 
plified portable telephone, PHS, the 20-byte fixed-length 
frames must necessarily be transmitted at a rate of one 
frame per 30 msec. On the other hand, it is not neces- 
sary to transmit the fixed-length frames in a silent period 
in which no audio is generated. In this case, on tfie 
receiving side, an audio decoder performs processing 
regarding It as silent, or outputs previously transmitted 
background sound. For transmission error, audio data 
provided with a 1-byte error detecting code is transmit- 
ted. Usually, the receiving side adopts the-so^alled 
muting method in which the audio data is discarded 
when an error is detected, that is. the audio decoder 
performs processing regarding it as silence. 

On the other hand, H.263 is an image coding 
method for generating vanat)le-length data, whose gen- 
erated bit rate varies depending on the nature of the 
input image and definition, visibility, and the number of 
image frames of the transmitted image, and so on. In 
H.263, variations in the bit rate due to retransmission 
control are adjusted by the coding control, too. 

When a 20-byte audio frame is Included once in 
one fixed-length multiplex frame, the length of one fixed- 
length multiplex frame Is 120 bytes. The length of an 
audio frame per one fix^-length multiplex frame may 
be set to 40 bytes or longer. In this case, however, it is 
necessary to store audio frames for storage in a multi- 
plex frame, causing delay in audio. 

Rg.1 is a diagram showing the structure of a fixed- 
length multiplex frame used in a multiplex transmission 
method according to a first embodiment of the present 
invention. In Fig.1, a header is provided at the head of 
the fixed-length multiplex frame. This header at least 
requires the functbn of distinguishing between multi- 
plexing with active speech and multiplexing with inactive 
speech. In this embodimerrt, tiie header itself has tiie 
error detecting code or tiie error correcting code. The 
lengtii of the header, including these codes, is selected 
to 1 byte. However, tiie length of the header can be 
decreased/increased depending on the number of pat- 
terns to be identified witii the header and tiie error 
detecting or error connecting capability, which is not lim- 
ited to the structure of this embodiment. 

The 1 20-byte fixed-lengtii multiplex frame minus 
tiie 1-byte header, or the 119-byte fixed-length field, is 
divided into two. a variable-lengtii sk>t 1 and a variable- 
lengtii slot 2. 

The variable-length slot 1 is for transmitting an 
audio data link frame, whose size is equal to that of the 
audio data link frame. 21 bytes. The audio data link 
frame is formed of a 20-byte audio frame and a 1-byte 
CRC code (an en-or detecting code). The size of the var- 
iable-length slot 1 is 0 byte when tiiere is no audio data 
link frame to be transmitted because speech is inactive. 

The variable-length slot 2 contains tiie entirety of a 
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data link frame storing video data, the frame number, 
the retransmission request code, and the error detect- 
ing code. The data link frame is one complete frame 
structure as one mass of data In error control. That is to 
say, a multiplex transmission system to which the 
embodiment is applied performs retransmission control 
of video data in data link frame units. In this embodi- 
ment, the size of the variable-length slot 2 changes 
depending on whether it is a speech period or a silent 
period. TTiat is to say, in the case of active speech, the 
variable-length slot 2 has a size (98 bytes) correspond- 
ing to the fixed-length field (119 bytes) minus the varia- 
ble-length slot 1 (21 bytes). In this case, the video data 
link frame is generated as a short frame having the 
same length of 98 bytes as the variable-length slot 2 
and transmitted in the variable-length slot 2. In the silent 
period, the variable-length slot 2 has the same size (119 
bytes) as the fixed-length field. In this case, the video 
data link frame is generated as a long frame having the 
same length of 1 19 bytes as the variable-length slot 2 
and transmitted in the variable-length slot 2. 

The video data link frame may be provided with var- 
ious codes depending on the method used for the 
retransmission control. For example, a reject code may 
be inserted as a retransmission request code only when 
an error occurs. Or, a data length code indicating the 
length of significant data may be provided, or stuffing 
(the operation of adding dummy bits in the empty part 
and removing them on the receiving side) may be per- 
formed, without a data length code, if an empty part 
takes place in the data field. 

As has been described so far, this embodiment 
changes the frame length of the video data link frame 
depending on whether it is a speech period or a silent 
period. This enables synchronization between the 
transmitting timing of the fixed-length multiplex frames 
and the transmitting timing of the data link fr£imes. The 
synchronization between the transmitting timing of the 
fixed-length multiplex frames and the transmitting timing 
of the data link frames herein means a state in which 
transmission of (a) complete Y (Y is a natural number: 
Y=1 in this embodiment) data link frame(s) is secured 
while X (X is a natural number : X=1 in this embodiment) 
fixed-length multiplex frame(s) Is (are) transmitted. In 
this case, considering the oveitieads accompanying the 
data link frame of video data (or computo- data), such 
as the data length, the frame number, the retransmis- 
sion request code, the error detecting code, and the 
retransmission efficiency determined by the en-or char- 
acteristics of the transmission path, two or more of the 
variable-length slots 2 in adjacent fixed-length multiplex 
frames may be combined and the data link frame for 
video nnay be structured to that length. Or, two video 
data link frames may be stored in three multiplex data 
link frames, or three video data link frames may be 
stored in four multplex frames. 

As described above, in this embodiment, ti^nsmis- 
sion of one complete data link frame is always secured 
while one fixed-length multiplex frame is transmitted. 



When a video data link frame is stored over a plurality of 
adjacent multiplex frames, it is necessary to identify 
said stored multiplex frames. In this embodiment each 
multiplex frame is completed and this provides a still 

5 higher tolerance for errors. 

Thus, in the first embodiment, the multiplex frames 
for carrying audio data and video data are of a fixed 
length and the data link frames for video have their 
frame lengths varied depending on whether it is in a 

10 speech period or a silent period, so as to acquire syn- 
chronization between tiie transmitting timing of tfie 
fixed-length multiplex frames and the transmitting timing 
of the data link frames. This enables constant transmit- 
ting timing for the data link frames, which prevents tfie 

15 ktoundaries between tiie data link frames from being lost 
in ttie event of a transmission en*or to provide robust- 
ness against errors. That is to say, unlike H.223 in which 
synchronization is acquired by using flags, bits are not 
shifted due to the "0" insertion, and even if slot synchro- 

20 nization is lost due to a transmission error, it is possible 
to see the next header at once, facilitating resynchroni- 
zation. This also enables effective utilization of the inac- 
tive speech periods to increase the transmission 
efficiency of video data, or the variable-length data 

25 employing retransmission. 

Although tiie first embodiment has explained multi- 
plexing of the audio data and the video data, other data 
such as tiie computer data may be transmitted as tiie 
variable-length data employing retransmission, or tiie 

30 audio data, the video data and the computer data can 
be multiplexed to provide tiie same effects. Although tiie 
first embodiment has explained the case in which tiie 
audio data link frame has a length of 21 bytes, the 
present invention is not limited to It. That is to say, the 

35 frame lengtii of the audio frame may be otherwise deter- 
mined. Audio data link frames provided with error-cor- 
recting redundant bits having different lengths, or audio 
data link frames coded to anotiier bit rate, e.g.. to 6.3 
Kbps. m£^ be used as well. 

40 Furtiiermore, while the audio data link frame in tiie 
first embodiment can take two kinds of lengths (21 bytes 
and 0 byte) in the speech period and the silent period, it 
may take three or more kinds of lengtfis. In this case, 
synchronization is acquired between the transmitting 

45 timing of tiie f txed-1 ength multiplex frames and tiie trans- 
mitting timing of the data link frames by adjusting tiie 
lengths of the variable-length slots 1 and 2 according to 
tiie length of the audio data link frame and sizing tiie 
video data link frame equal to tiie lengtii of the variable- 

50 length slot 2. 

(Second Preferred Embodiment) 

Fig.2 is a diagram showing a structure of a fixed- 
55 length multiplex frame used in a multiplex transmission 
method according to a second embodiment of tiie 
present invention. This embodiment adopts the struc- 
ture of tiie ARQ frame of PIAFS as a structure of tiie 
fixed-length multiplex frame. That is to say. in tiiis 
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embodiment, the video data as the variable-length data 
employing retransmission and the audio data as the 
fixed-length data not employing retransmission are 
stored in the user data field of the ARQ frame in PIAFS. 

As has been explained above, the ARQ frame of 
PIAFS is a data link frame formed of the frame type 
identification field for storing the frame type identifica- 
tion code, the user data field for storing user data, the 
user data length indication field indicating the data 
length of the significant data in the user data field, the 
error control field for storing the frame number and the 
request frame number used in the retransmission con- 
trol, and the error correcting code field for storing the 
error correcting code for detecting error in the entire 
ARQ frame. 

In this embodiment, as shown in Fig.2{a), the user 
data field is divided into a data field for variable-length 
data employing retransmission and a data field for fixed- 
length data not employing retransmission to store the 
audio data in the user data field. These fields are previ- 
ously agreed between transmitting and receiving sides. 

When audio data is coded by using G.723. audio 
data is generated at 20 bytes/30 msec. In the G.723 
system, however, no audio data is generated tf it is silent 
during the unit time of 30 msec. When no audio data is 
generated, transmission is made by using the frame 
shown in Fig.2(b) in this enrtxKliment 

When audio data to be multiplexed exists, the trans- 
mitting side performs the transmission by using the 
frame of Fig.2(a). That is to say, the transmission is 
made with the video data stored in the data field for var- 
iable-length data employing retransmission and the 
audio data provided with a CRC2, an error detecting 
code, stored in the data field for fixed-length data not 
employing retransmission. On the other hand, with no 
audio data to be multiplexed, the transmitting side con- 
ducts the transmission by using the frame of Fig.2(b). 
That is to say, it performs the transmission with only the 
video data stored in the data field for variable-length 
data employing retransmission. Stored in the frame type 
identification field is a code indicating whether audio 
data is multiplexed in the user data field and stored in 
the user data length indication field is the data length of 
the video data. Stored in the error control field is the 
frame number and the request frame numkier required 
to conduct the retransmission control defined in PIAFS. 
Stored in the error correcting code field is the CRC1. 
which is the error detecting code for en^or detection in 
the dotted parts in Fig.2(a) and Fig.2(b). i.e., the frame 
type identification field, the user data length indication 
field, the video data having the length shown in the user 
data length indication field and the en^or control field. 

On receiving the fixed-length multiplex frame, the 
receiving side first reads the frame type identification 
code in the frame type identification field to determine 
whether audio data is multiplexed in the user data field. 
When audio data is multiplexed as shown in Fig.2(a), it 
extracts the audio data and the CRC2 from the no- 
retransmission-errploying fixed-length data field. Then 



it detects error in the audio data by using the CRC2. 
When detecting an error, it discards the audio data, and 
when detecting no error, it gives the audio data to the 
upper layer. 

6 The receiving side applies the transmission control 
defined in PIAFS to the data left after the audio data has 
been extracted. First, it reads the video data length from 
the user data length indication field to determine tlie 
area to be assigned to the CRC1 in the enor correcting 

10 code field and performs error detection. In Fig.2(a) and 
(b), the dotted parts correspond to the area error- 
detected by the CRC1 . tf an en^or is found, the data from 
which the audio data has been extracted is discarded. 
This way. in the second embodiment, fixed-length 

15 data not employing retransmission control is multiplexed 
in the user data field of the ARQ frame defined in 
PIAFS. This enables multiplex transmission of video 
data and audio data without deteriorating the frame syn- 
chronization function providing a high tolerance for en^or 

20 realized in PIAFS. 

In the second embodiment, the information indicat- 
ing whether audio data is multiplexed or not is Included 
in the frame type identification code. However, the 
receiving side may detennine that a data field for fixed- 

25 length data not employing retransmission exists only 
when the video data length incficated in the user data 
length indication field Is shorter than the length of the 
field for variable-length data employing retransmission 
in Fig.2(a) to conduct the error detection of the audio 

30 data. Then if no error is found, it will give the audio data 
to the upper layer. This eliminates the necessity of 
including the information showing whether audio data is 
multiplexed in the frame type identification code, then 
the frame type identification code defined in PIAFS can 

35 be used as it is. 

Although the second embodiment has explained 
multiplexing and transmitting of the audio data and the 
video data, other data such as the computer data may 
be transmitted as the variable-length data employing 

40 retransmission, or the audio data, the video data and 
the computer data can be multiplexed to provide the 
same effects. The audio data defined in PIAFS is 
applied to the retransmission control, but it is not limited 
to it The retransmission control with less delay may be 

45 applied to it. 

Although the second embodiment has explained 
the case in which the audio frame has a length of 20 
bytes, the present invention is not limited to It. That is to 
say, the frame length of the audio frame may be set to 

50 another value. Audio data link frames provided with 
error-correcting redundant bits of different lengths, or 
audio frames coded to another bit rate, e.g.. 6.3 Kbps. 
may be used as well. 

55 (Third Preferred Embodiment) 

In the first and second embodiments described 
above, the following problem arises when audio data 
and other data are multiplexed. Rg.3 shows an example 
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of multiplexing in which audio data coded by G.723 is 
multiplexed with video data and transmitted through the 
32-Kbps transmission path of PHS, a simplified portable 
telephone, with the multiplex frame length set to 120 
bytes. As has been stated above, G.723 codes audio at 5 
a bit rate of 5.3 Kk»ps. wherein the audio signal is coded 
in 30-msec unit to form a fixed-length frame of 20 bytes. 
This fixed-length frame of 20 bytes is undividable and 
must necessarily be transmitted at a rate of one per 30 
msec in a speech period. That is to say, in the case of w 
PHS, an audio frame of 20 bytes must be multiplexed 
every time 120-byte data is transmitted. In a silent 
period without audio, it is not necessary to transnrut the 
fixed-length frame. In this case, the audio decoder on 
the receiving side performs processing for silence, or ^5 
outputs previously transmitted background sound. 
When the multiplex frame length is set to 120 bytes, the 
size of the video slot differs in a speech period and a 
silent period, as shown in Fig.4. 

In the first and second embodiments, a slot must 20 
include one complete ARQ frame (i e.. a data link 
frame). If the ARQ frame size is set equal to the slot size 
to improve the transmission efficiency, the size of the 
ARQ frame for video differs in a speech period and a 
silent period. Then, if a video frame transmitted in a 25 
silent period fails, it can not be retransmitted in a video 
slot in a speech period. The retransmission must be 
performed in the next silent period, or it must be per- 
formed by inserting a silent frame into a speech period 
in a forced manner. When the retransmission is made in 30 
the silent period coming next, a delay occurs in the 
retransm'ssion data. This data delay exerts fatal effect 
especially on the transmission of the video data which 
requires real-time performance. When a silent frame is 
forced into a speech period, the transmission error only 35 
in the variable-length data employing retransmission 
causes audio to be muted, as if an error existed in 
audio, providing adverse effects on the quality of audio. 
The third embodiment is to solve these problems, which 
will now be explained refening to the drawings. 40 

Fig.5 is a diagram showing the structure of a fixed- 
length multiplex frame used in a multiplex transmission 
method according to the third embodiment of the 
present invention. In Fig.5, the fixed-length multiplex 
frame used in this embodiment includes a header, a 45 
fixed-length field, and a fixed-length slot. The fixed- 
length field is divided into two, a variable-length slot 1 
and a variable-length slot 2. As in the first or second 
embodiment, the variable-length slot 1 contains a 20- 
byte audio frame plus a 1-byte CRC. A video data link so 
frame is transmitted in the variable-length slot 2. whose 
size increases in a silent period, as has been explained 
in the first and second embodiments. The entire length 
of the fixed-length multiplex frame is selected to be 120 
bytes. 55 

The length of the fixed-length slot is set equal to or 
longer than the length of the fixed-length field. The 
retransmission of the variatDle-length slot 2 is made in 
the fixed-length slot in this embodiment. Hence, consid- 
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ering reduction of efficiency, it is desirable to set the 
length of the fixed-length slot to be as dose to the length 
of the fixed-length field as possible. If the l&igth of the 
fixed-length field and the length of the fixed-length slot 
are equal, it is possible to retransmit a data link frame 
transmitted in the fixed-length slot by using the variable- 
length slot 2 in a silent period. Herein, it is impossible to 
set the length of the fixed-length field equal to that of the 
fixed-length slot, for the header length is of an odd 
number of 1 byte. Accordingly, in this embodiment, the 
fixed-length field is set to 59 bytes and the fixed-length 
slot is set to 60 bytes. 

This embodiment is now be explained in more detail 
in an example in which audio data coded by 0.723 and 
video data coded by H.263 are multiplexed, like in the 
first emtxxiiment. 

In a speech period, in this embodiment, a video 1 
frame with a 37-byte data link frame length is transmit- 
ted in the variable-length slot 2 and a video 2 frame with 
a 60-byte data link frame length is transmitted in the 
fixed-length slot. It is then repeated, where retransmis- 
sions of the frames are conducted by using the respec- 
tive slots. 

On the other hand, in a silent period, in this embod- 
iment, the video 1 data link frame is set to 59 bytes and 
transmitted in the variable-length slot 2 and the video 2 
frame with a 60-byte data link frame length is transmit- 
ted in the fixed-length slot. 

Next, the retransmission is described. When a data 
link frame transmitted in a silent period has an error, it is 
transmitted in the same type of slot as long as the silent 
period continues. If speech is g^erated at the time of 
retransmitting a long frame transmitted in a silent 
period, the long frame of the video 1 frame is retransmit- 
ted in the fixed-length slot, instead of transmitting a new 
video 2 frame. If the speech becomes inactive at the 
time of retransmitting a video 1 frame transmitted in the 
previous speech period, it is transmitted in the long 
frame of the video 1 with a data length specified or stuff- 
ing. 

Thus, according to the third embodiment, the length 
of the data link frame of video data in the fixed-length 
field is increased/decreased depending on the length of 
the audio frame to keep the transmitting timing of the 
fixed-length nnuttiplex frames and the transmittng timing 
of the data link frames in synchronization. Then the 
boundaries between the data link frames will not be lost 
due to a transmission en'or, providing improved en^or 
robustness. Furthermore, a long frame of the variable- 
length slot 2 can be retransmitted by using a fixed- 
length slot without waiting for the next silent period to 
come, enabling retransmission without video delay. 

Although the third embodiment has described a 
multiplex transmission for transmitting audio data and 
video data, other data such as computer data may be 
transmitted as the variable-length data employing 
retransmission. It is also possible to multiplex and trans- 
mit audio data, computer data and video data by trans- 
mitting the video data in the variable-length slot 2 and 
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transmitting the computer data in the fixed-length slot, 
for example, through the silent periods can not be effec- 
tively utilized in this case. Considering the overheads 
accompanying the data link frame of the video data (or 
the computer data), such as the data length, the frame 
number, the retransmission request code, the error 
detecting code, and the retransmission efficiency deter- 
mined by the error characteristics of the transmission 
path, two or more variable-length slots 2 in adjacent 
fixed-length multiplex frames may be combined and the 
data link frame for video data may be structured to that 
length. 

Although the third embodiment has explained the 
case In which the audio frame has a length of 20 bytes, 
the present invention is not limited to it. That is to say, 
the frame length of the audio frame may take other val- 
ues. Audio data linkf rames provided with redundant bits 
for error correction, or audio frames coded to another bit 
rate, e.g.. 6.3 Kbps. may be used as well. 

(Fourth Preferred Embodiment) 

Fig.6 is a diagram showing the structure of a fixed- 
length multiplex frame used in a multiplex transmission 
method according to a fourth embodiment of the 
present invention. In Fig.6, this embodiment alternately 
and continuously transmits a 60-byte first fixed-length 
multiplex frame containing a header, a variable-length 
slot 1. and a variable-length slot 2 and a second fixed- 
length nrruttiplex frame having the same length as the 
first f ixed-tength multiplex frame and containing a fixed- 
length slot in the part other than the header. The inter- 
nal structure of the first fixed-length multtplex frame is 
the same as that of the fixed-length multiplex frame of 
the first ennbodiment. However, note that the frame 
length is half of that of the first embodiment (60 bytes). 
The internal structure of the second fixed-length multi- 
plex frame includes the fixed-length slot of the third 
embodiment and the header. 

In this embodiment the fixed-length data not 
employing retransmission (e.g.. audio data) is transmit- 
ted by using the variable-length slot 1, the variable- 
length data employing retransmission (e.g., video data) 
by using the variable-length slot 2, and the variable- 
length data employing retransmission (e.g., video data) 
by using the fixed-length slot. The variable-length data 
employing retransmission is retransmitted, when an 
error occurs, in the same manner as that described in 
the third embodiment, which is not explained again 
here. 

(Fifth Prefen-ed embodiment) 

Fig. 7 is a diagram showing the structure of a fi^- 
length multiplex frame used in a multiplex transmission 
method according to a fifth embodiment of the present 
invention. In Fig.7, like the fourth emtxxliment. this 
embodiment transmits a 60-byte first fixed-length multi- 
plex frame containing a header, a variable-length slot 1 , 
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and a variable-length slot 2 and a second fixed-length 
multiplex frame having the same length as the first 
fixed-length multiplex frame and containing a fixed- 
length slot in the part other than the header in an alter- 

5 nate and continuous manner. The first fixed-length mul- 
tiplex frame has the same internal structure as the fixed- 
length multiplex frame of the second embodiment. The 
second fixed-length multiplex frame has the same inter- 
nal structure as the PI AFS ARQ frame. 

10 In this embodiment, fixed-length data not employing 
retransmission (e.g., audio data) is transmitted by using 
the variable-length slot 1, variable-length data employ- 
ing retransmission (e.g., video data) is transmitted by 
using the variatile-length slot 2, and variable-length data 

15 employing retransmission (e.g.. video data) is transmit- 
ted by using the fixed-length slot. The variable-length 
data employing retransmission is retransmitted, when 
an error occurs, in the same manner as that In the third 
embodiment, which is not explained here again. 

20 The fourth and fifth embodiments have shown 
examples in which a 60-byte first fixed-length multiplex 
frame including a variable-length slot 1 for storing a 20- 
byte audio frame and a 60-byte second fixed-length 
multiplex frame including no audio frame are alternately 

25 transmitted to transmit 20-byte audio data per 120 
bytes. In this case, the frame length of the variable- 
length data employing retransmission Includes 40 bytes 
and 60 bytes. 

As the length of the retransmission-employing vari- 

30 able-length data frame closely relates to the transmis- 
sion efficiency, a still larger frame length may be 
suitable. Accordingly, as shown in Fig.8(a) and Fig.9(a). 
a 80-byte first fixed-length multiplex frame including a 
variable-length slot 1 for storing a 20-byte audio frame 

35 and a 80-byte second fixed-length multiplex frame 
including no audio frame may be transmitted in such a 
pattern that they appear at a ratio of 2:1 . 

Or. as shown in Fig.8(b) and Fig.9{b). a 80-byte first 
fixed-length multiplex frame including a variable-length 

40 slot 1 capable of storing two 20-t>yte audio frames and a 
80-byte second fixed-length multiplex frame including 
no audio frames may be transmitted in such a pattern 
that they appear at a ratio of 1 2. 

In addition, also possible are a pattern in which a 

45 90-byte first fixed-length multiplex frame including a 20- 
byte audio frame and a 90-byte second fixed-length 
multiplex frame including no audio frame appear at a 
ratio of 1 :3. or a pattern in which a 100-byte first fixed- 
length multiplex frame including a 20-byte audio frame 

50 and a 100-byte second fixed-length multiplex frame 
including no audio frame appear at a ratio of 1 :5. 

In any of the above-described implementations. 20- 
byte audio frames can be transmitted per 120 bytes. In 
any of the cases, the variable-length slot 2 forms a long 

55 frame when no audio frame is generated in a silent 
period. When an error occurs in this long frame, It is 
retransmitted, if the silent period is continuing, by using 
the variable-length slot 2 forming a long frame in the first 
fixed-length multiplex frame, or it is retransmitted, if 
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speech takes place, by using the fixed-length slot of the 
second fixed-length multiplex frame. 

Thus, according to the fourth and fifth embodi- 
ments, while the silent periods are effectively utilized, 
synchronization is acquired between the transmitting 
timing of the data linkframes and the transmitting timing 
of the fixed-length multiplex frames by increas- 
ing/decreasing the data fink frame length of the video 
data, or the variable-length data employing retransmis- 
sion. TTiis realizes video and audio multiplex transmis- 
sion having error robustness and high efficiency. 
Furthermore, the retransmission of a long frame of the 
variable-length slot 2 can be made, without waiting for 
the next silent period to come, by using the fixed-length 
slot. This enables retransmission without delay of video. 

Although the fourth and fifth embodiments have 
described the multiplex transmission for transmitting 
audio data and video data, other data such as computer 
data may be transmitted as the variable-length data 
employing retransmission. In this case, the same data 
link frames having the same header may be used for the 
first fixed-length multiplex frame in which the variable- 
length slot 2 forms a long frame and the second fixed- 
length multiplex frame. It is also possible to multiplex 
and transmit audio data, computer data and video data 
by transmitting the video data in the variable-length slot 
2 and transmitting the computer data in the fixed-length 
slot, through the silent periods can not be effectively uti- 
lized in this case. Considering the overheads accompa- 
nying the data link frame of the video data (or the 
conrputer data), such as the data length, the frame 
number, the retransmission request code, and the error 
detecting code, arxj the retransmission efficiency deter- 
mined by the error characteristics of the transmission 
path, two or more variable-length slots 2 in adjacent 
fixed-length multiplex frames may be combined and a 
data link frame for the video data may be structured to 
that l^gth. 

Although the fourth and fifth embodiments have 
explained the case in which the audio frame has a 
length of 20 bytes, the present invention is not limited to 
it That is to say, the frame length of the audio frame can 
take other values. Audio data link frames provided with 
redundant bits for error correction, or audio frames 
coded to another bit rate. e.g. , to 6.3 Kbps, may be used 
as well. 

(Sixth Preferred Embodiment) 

The first to fifth embodiments have shown imple- 
mentations in which the video data is transmitted as the 
variable-length data employing retransmission. Now, an 
implementation in which common computer data Is 
transmitted as the variable-length data employing 
retransmission will be considered. 

Fig. 10 Is a layer diagram showing the structure of a 
multiplex transmisson device according to a sixth 
embodiment of the present invention. The multiplex 
transmission device of this embodiment is so con- 



structed as to perform the transmission control accord- 
ing to that suggested in the above descrived PIAFS. 

Fig.11 is a diagram showing a first example of 
structure of a fixed-length multiplex frame used In the 

s multiplex transmission device of Fig. 10. In the fixed- 
length multiplex frame of Fig.11, like that in the first 
embodiment, a data link frame for the variable-length 
data employing retransmission and a data link frame for 
the fixed-length data not employing retransmission are 

10 multiplexed together with the header containing the 
information showing the type of the multiplexing. 

On the transmitting side, as shown in Fig. 1 1(a). the 
PIAFS data link processing block 1803 produces a 
PIAFS ARQ frame in which the significant data (the var- 

15 iable-length data employing retransmission) length in 
the user data is increased/decreased depending on 
presence/absence of a data link frame for the fixed- 
length data not employing retransmission to be multi- 
ple^ in the fixed-length multiplex frame. More specifi- 

20 cally. the PIAFS data link processing block 1803 limits 
the length of the significant data in the user data under 
the maximum length of the user data minus the length of 
the data link frame for the fixed-length data not employ- 
ing retransmission to be multiplexed and the header 

25 length. The converting block 1802 eliminates the 
dummy bits in the user data to convert it into a data link 
frame for the variable-length data employing retransmis- 
sion having a reduced PIAFS user data field. The multi- 
plexing block 1801 multiplexes a data link frame of fixed- 

30 length data not employing retransmission and the data 
link frame of the variable-length data employing retrans- 
mission and adds the header to constitute a fixed-length 
multiplex frame. (Refer to Rg.l 1(b).) 

The length of the fixed-length multiplex frame is set 

35 to the same size as the PIAFS ARQ frame length In 
PHS data oommunication (80 bytes). In this case, when 
they are transmitted by the method shown in the fifth 
embodiment (refer to Fig.9). the length of the fixed- 
length multiplex frames can be set to 80 bytes. When 

40 the fixed-length multiplex frame has the same frame 
length as the PIAFS ARQ frame length in the PHS data 
oommunication. the frame timing of the data link frames 
of the variable-length data employing retransmission 
can have the same cyde and the same rate as the 

45 frame timing of PIAFS in the PHS data oommunication. 
On the receiving side, the multiplex block 1806 
demultiplexes the received fixed-length multiplex frame 
into the data link frame of the fixed-length data not 
employing retransmission and the data link frame of the 

50 variable-length data employing retransmission, and 
gives the former to the audio adaptation block 1809 and 
the latter to the converting block 1807. As shown in 
Fig. 11(c), the converting block 1807 inserts the fill bits 
"1" to the part the number is shorter than 80, between 

55 the user data field and the error detecting code to size 
the entirety to 80 bytes, and gives it to the PIAFS data 
link processing block 1808 for the PHS data communi- 
cation. Thus the received data can be easily converted 
into a PIAFS ARQ frame for PHS data communication, 
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then a processing block lor common PHS data commu- 
nication can process the data. 

Fig. 12 is a diagram showing a second example of 
structure of fixed-length multiplex frame used in the mul- 
tiplex transmission device of Fig.10. Like those in the 
second embodiment, the fixed-length multiplex frames 
of Fig.l2 have the same structure as the ARQ frame of 
PIAFS. wherein the audio data, or the fixed-length data 
not employing retransmission, is embed and multi- 
plexed in the user data field. The length of the fixed- 
length multiplex frame is the same as the length (80 
bytes) of the PIAFS ARQ frame in the PHS data com- 
munication. As shown in Fig. 12(a), the frame format of 
the fixed-length multiplex frame other than the user data 
field is the same as that of PIAFS in the PHS data com- 
munication. Stored in the user data length irxJication 
field is the length of the significant variable-length data 
employing retransmission. 

On the receiving side, the multiplexing block 1806 
extracts the fixed-length data not employing retransmis- 
sion (audio data) from the fixed-length multiplex frame 
received and gives it to the audio adaptation block 1809. 
The multiplex block 1806 gives the fixed-length multi- 
plex frame 1o the converting block 1807 as it Is As 
shown in Fig.12(b), the converting block 1807 entirely 
fills the field for the fixed-length data not errploying 
retransmission with the fill bits T and gives it to the 
PtAFS data link processing block 1808 for PHS data 
communication. This allows the received data to be eas- 
ily convened into a PIAFS ARQ frame for PHS data 
communication, which allows a processing block for 
performing common PHS data comnnunication to proc- 
ess the data. 

(Seventh Preferred Embodiment) 

When the multiplex frame has a fixed length, rt is 
desirable that the frame length corresponds to the audio 
frame generation cycle divided by an integer. For exam- 
ple, G.723 generates an audio frame with a cyde of 30 
msec. If it is transmitted at 32 Kbps, the cycle is 120 
bytes. Accordingly, it is desirable that the multiplex 
frame length is set to 120 bytes, 60 bytes, or 40 bytes. 
However, since the retransmission effk)iency of ARQ 
depends on the relation between the en'or characteris- 
tics and the frame length, these multiplex frame lengths 
are not necessarily suitable. In PIAFS, it is possible to 
select the communication protocol by in-band negotia- 
tion. When the AV nnultlplex communication is selected, 
a multiplex frame length of 80 bytes enables continuous 
utilization of the synchronization established by PIAFS. 
This improves the transmission efficiency and reduces 
the processing procedure. 

When a low-bitrate-coded 25-byte audio frame gen- 
erated at the rate of one frame per 120-byte transmis- 
sion time is carried on a multiplex frame with a fixed 
length of 80 bytes (refer to Fig. 13(a)). one multiplex 
frame having no multiplexed audio frame appears in 
every three multiplex frames. This causes jitter of 40 



bytes in the time intervals between the audio frames. 
(Refer to Fig. 13(b).) In this specification, the variation in 
the relative time intervals between the audio frames 
(repeating 80-byte intervals and 160-byte Intervals) with 

5 respect to the expected cycle (1 20 byte cycle) is defined 
as jitter. That Is to say, the difference (the 40-byte inter- 
val) between the relatively short one of the audio frame 
intervals (the 80-byte interval) and the expected cycle 
(120-byte cycle) and the difference (40 bytes) between 

10 the relatively long time interval (the 160-byte interval) 
and the expected cyde (the 120-byte cyde) are 
regarded as jitter. A seventh embodiment is directed to 
a method for absorbing the jitter on the side receiving 
the multiplex frames. 

15 " Rg.13 is a diagram for use in describing the jitter 
absorbing method according to the seventh embodi- 
ment of the present Invention. The multiplex information 
MC in Fig.13(a) indudes numbers indicating the multi- 
plex structure of the multiplex frames. The relation 

20 between the numbers and the multiplex structure is sig- 
naled from the transmitting side to the receiving side 
prior to the transmission of AV signal. It is now assumed 
tiiat the relation between the multiplex structure and the 
multiplex information is specified by using the multiplex 

25 code table shown in Fig. 14. 

As shown in Fig. 14. MC«1 indicates indusion of a 
25-byte audio frame and a 51 -byte video frame and 
MC»2 indicates indusion of a 76-byte video frame 
alone. Each multiplex frame has a fixed length of 80 

30 bytes. Accordingly, two 1 20-byte-cyde audio frames are 
included in every three multiplex frames. 

In the conventional nnanner, as stated above, in the 
case of the jitter of 40 bytes, for example, a received 
audio frame is accumulated in a buffer for the 40-byte 

35 transmission time or longer and reading of the audio 
frame is started when the 40-byte transmission time has 
passed, or later. Subsequently, the audio frames 
received before elapse of the 120-byte time are accu- 
mulated in the buffer so that the audio frames can be 

40 outputted with a 120-byte cycle. However, this method 
causes a delay of 40-byte transmission time on the 
receiving side. (Refer to Fig. 13(b).) 

On tiie multiplex frame stream, as shown in 
Rg.13(a], an audio frame multiplexed from the next 

45 audio frame at a relatively short time interval (the 80- 
byte interval) and an audio frame multiplexed from ttie 
next audio frame at a relatively long time interval (the 
160-byte interval) alternately repeat. In this embodi- 
ment, the audio frame separated from tiie next audio 

50 frame at the relatively short time interval on the multi- 
plex frame stream is outputted inrvnediately after ttie 
reception without being stored in tiie buffer. After that, 
the audio frames received before elapse of the 120-byte 
time are accumulated in the txjffer so that the audio 

55 frames can be outputted on the 120-byte time cycle. 
This eliminates the delay of the 40-byte time which the 
conventional art has been suffered from. (Refer to 
Fig.13(c).) 

This way, the seventh embodiment enables elimina- 
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tion of jitter without causing delay when 120-byte cyde 
audio is multiplexed and transmitted on 80-byte fixed- 
length multiplex frames. Particularly, when the jitter 
absorbing method of this embodiment is comt>ined with 
the PIAFS devised as a transmission control method for 5 
transmitting computer data through the radio channel of 
PHS having a bit rate of 32 Kbps, the audio data can be 
multiplexed and transmitted without causing jitter and 
delay while adopting the multiplex frame length provid- 
ing good transmission efficiency. 10 

Methods for identifying the audio frame separated 
from the next audio frame in a relatively short time on 
the multiplex frame stream include the following meth- 
ods, for example: A first method is to receive a plurality 
of multiplex frames first to acquire synchronization to is 
the pattern of the multiplex frame stream; and a second 
method is to settle between the transmitting side and 
the receiving side in advance that a perdetermined mul- 
tiplex frame (i.e.. the multiplex frame containing an 
audio frame separated from the next audio frame in a 20 
relatively short time) always comes first. 

It depends on the processing capability on the 
receiving side and the multiplex structure to which of the 
timings in the multiplex structure of Fig.13(a) the timing 
on the left end in Rg. 13(b) and (c) con^esponds. It may 25 
be immediately after the reception of an audio frame, or 
may be after conpletion of the reception of the multiplex 
frame. 

When the relation between the multiplex structure 
and the multiplex information is defined by using the 30 
multiplex code table in Fig. 14 with one multiplex frame 
having one header, the timing will be like that shown in 
Fig. 13(a). (b). (c). When the relation between the nrujlti- 
plex structure and the multiplex information is defined 
by using the multiplex code table of Fig. 15. the timing 35 
will be like that shown in Fig. 16(a). (b). (c). When the 
audio data and video data are multiplexed in this order 
in a multiplex frame with each multiplex frame provided 
with two multiplex headers at the head and the end. the 
timing will be like that shown in Rg. 17(a), (b). (c). This is 40 
for the following reason: If data is to be outputted imme- 
diately after the arrival of the audio data as shown in 
Fig. 18 (a), (b), (c), the output timing will be like that 
shown in Fig. 18 (c), which causes jitter. Because the 
output is provided after the arrival of the multiplex as 
header at the end when the multiplex header at the 
head is errored and the multiplex header at the end 
when the multiplex header at the head is errored and 
the multiplex header at the end is correctly received. As 
can be seen by refening to the diagrams, the present so 
invention can provide the same effects in any of the mul- 
tiplex structures. 

The seventh embodiment has shown an example in 
which no synchronization flag is provided at the head of 
the multiplex frame. However, the present invention can ss 
be applied to provide the effects described in the 
embodiments as long as the audio frames are transmit- 
ted In multiplex frames of different cycle, even when the 
multiplex frames always have a synchronization flag at 
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the head without utilizing synchronization by frames, or 
when the multiplex frames are defined by another for- 
mat, or when the multiplexed information include data or 
control information. 

Although the seventh embodiment has explained 
the case in which the audio frame has a length of 25 
bytes, the present invention is not limited to it. That is to 
say, the frame length of the audio frame can take other 
values. Audio frames provided with redundant bits for 
error correction, or audio frames coded to another bit 
rate, e.g.. 6.3 Kbps, may be used as well. 

(Eighth Preferred Embodiment) 

The seventh embodiment has shown the following 
two methods as methods for identifying an audio frame 
separated from the next audio frame at a shorter time 
interval on a stream of the multiplex frames: 

(1) A method of receiving a plurality of multiplex 
frames first and adjust the synchronization to the 
pattern of the multiplex frame stream. (Refen^ed to 
as a first method, hereinafter.) 

(2) A method of previously settling between the 
transmitting side and the receiving side that a cer- 
tain multiplex frame (i.e., a multiplex frame including 
an audio frame separated from the next audio 
frame in a shorter time) always comes first. 
(Referred to as a second method, hereinafter). 

The above methods, however, introduce the follow- 
ing prot)lems. 

In the case of the first method, even if the plurality 
of multiplex frames received first include audio frames, 
the audio frames can not be decoded until syncftroniza- 
tion has been acquired to the pattern of the multiplex 
frame stream. That is to say. the first active speech can 
not be reproduced in the first method. Furthermore, in 
the first method, if a silent frame is generated first, a sin- 
gle audio frame can not be reproduced until active 
speech frames are successively generated. On the 
other hand, in the second method, if a transmission 
enor occurs in the multiplex frame, it becomes impossi- 
ble to know which is the first multiplex frame. 

In any of the methods, if a reference audio frame is 
missed, the next audio frame does not anrive even when 
a 120-byte time has passed. However, the audio 
decoder recognizes it just as a silent period and contin- 
ues decoding. As a result, the audio sounds strange to 
listeners but the decoder can not recognize it. 

Accordingly, an eighth embodiment of the invention 
realizes a method for enabling identification of a refer- 
ence audio frame prior to transmission/reception of the 
audio frames. 

The eighth embodiment uses the multiplex code 
table shown in Fig.19 to define the relation between the 
multiplex structure and the multiplex information. 
Fig. 1 3(d) shows an example of a multiplex frame stream 
used in the eighth embodiment. In Fig.19, as the multi- 
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plex information for the multiplex frames having the 
same structure and in \A/hich an audio frame and a video 
frame are multiplexed, the multiplex information MC=1 
Is assigned to the multiplex frame separated from the 
next audio frame in a shorter time, and the multiplex 
information MC=2 is assigned to the multiplex frame 
separated from the next audio frame in a longer time. 
The multiplex information MC=3 is assigned to the mul- 
tiplex frame containing only a video frame. 

The relation between the frame numbers and the 
multiplex structures is defined on the transmitting side 
and transferred to the receiving side prior to a transmis- 
sion of the audio and video data. Accordingly, the 
receiving side easily recognizes, at the time of receiving 
the multiplex information (MC), that different, two kinds 
of multiplex information are assigned to the multiplex 
frames having the same kind of multiplex structure. For 
example, when it is predetermined to use an audio 
frame multiplexed in a multiplex frame with multiplex 
information of a smaller MC number as a reference 
audio frame, the reference audio frame is instantly iden- 
tified on the reception of the multiplex information In the 
multiplex frame. 

(Ninth Preferred Embodiment) 

It is also possible that a transmitting terminal does 
not support the fonction described above. In this case, 
however, it is possible to absorb the audio jitter if the 
receiving terminal has the functions of both of the sev- 
enth and eighth embodiments. The processing on the 
receiving terminal side in this case is shown in the ffow 
chart of Fig.20. 

In Fig.20, the receiving terminal receives, prior to a 
transmission of the audioMdeo data, the multiplex mul- 
tiplex code table describing the relation between the 
multiplex information and the multiplex structures (Step 
S201). Next, the receiving terminal analyzes the 
received multiplex infonnation (MC) to determine 
whether different MC numbers are provided to multiplex 
frames having the same multiplex structure containing 
muttiptexed audio frames (Step S202). 

If the determination in Step S202 is YES. the flow 
branches to the processing described in the eighth 
embodiment. That Is to say, the receiving terminal waits 
for reception of a multiplex frame (Step S203). When 
receiving one, it analyzes the multiplex structure of the 
received multiplex frame from the multiplex information 
(MC) Included in the multiplex frame (Step S204). Next, 
the receiving terminal determines on the basis of the 
result of the analysis in Step S204 whether the received 
multiplex frame is a multiplex frame having an audio 
frame multiplexed (Step S205). K no audio frame is mul- 
tiplexed in the received multiplex frame, the receiving 
terminal returns to Step S203 to wait for reception of the 
next multiplex frame. On the other hand, if the received 
multiplex frame contains an audio frame multiplexed 
therein, the receiving terminal determines whether that 
audio frame is a reference audio frame (Step S206). If 



the multiplexed audio frame is not a reference audio 
frame, the receiving terminal returns to Step S203 to 
wait for reception of the next multiplex frame. On the 
other hand, if the multiplexed audio frame is a reference 

5 audio frame, the receiving terminal starts the audio j itter 
absorbing processing (Step S207). In this audio jitter 
absorbing processing, the audio frame received this 
time is transferred as a reference frame to the audio 
decoder without delay and the following audio frames 

10 are transfen-ed to the audio decoder every time a 120- 
byte time passes. Audio frames received before the 
120-byte time has passed are tenporarily stored in a 
buffer. 

On the other hand, if the determination iri Step 

15 S202 is NO, the flow nxives to the processing described 
in the seventh entKxJiment. That is to say, the receiving 
terminal waits for reception of a multiplex frame (Step 
S208). When receiving one. it analyzes the multiplex 
structore of the received multiplex frame from the multi- 

20 plex information (MC) included in that multiplex frame 
(Step S209). Next, tiie receiving terminal determines on 
the basis of the resuR of the analysis in Step S209 
whetiier tiie received multiplex frame is a multiplex 
frame containing a multiplexed audio frame (Step 

25 S210). If no audio frame is multiplexed in the received 
multiplex frame, the receiving terminal returns to Step 
S208 to wait for rec^ion of the next multiplex frame. 
On the other hand, if an audio frame is multiplexed in 
tiie received multiplex frame, tiie receiving terminal 

30 determines whether a timer 1 has been started or not 
(Step S211). This timer 1 is started when a multiplex 
frame containing a multiplexed audio frame is received, 
and stopped when a multiplex frame containing a multi- 
plexed audio frame is received again before the time- 

35 out. The time-out time of tiie timer 1 is set larger than 
the shorter multiplex frame inten^al between audio 
frames in the multiplex frame stream, and smaller than 
tiie longer multiplex frame inten^al between the audio 
frames. Specifically, the timer 1 is started in Step S212 

40 and stopped in Step S21 4. 

Now. suppose that, on the receiving terminal side, 
the interval between tiie audio frame received first and 
the audio frame received second is short and the inter- 
val between tiie audio frame received second and the 

45 audio frame received third is long. In this case, the timer 
1 is started in response to tiie reception of the first audio 
frame (Step S212). Then the second audio frame is 
received before the timer 1 expires, in response to 
which the timer 1 is stopped (Step S213). Subsequentiy, 

50 tiie timer 1 Is started in Step 8212 again to start the 
counting operation again from a O-dear state. Next, the 
timer 1 time-outs before tiie third audio frame Is 
received. Accordingly, the receiving terminal deter- 
mines that the timer 1 has expired at the time when the 

55 tilird audio frame is received (Step S2 1 3) and starts the 
audio jitter absorbing processing (Step S207). 

Thus, like the seventh embodiment, the eighth and 
ninth entKxJiments allow the 120-byte-cycle audio to be 
multiplexed and transmitted in the 80-byte multiplex 
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frames without jitter and delay. Furthermore, it reduces 
the number of initial audio frames which are not repro- 
duced, as conrpared with the seventh embodiment. 
When the jitter absorbing methods of these embodi- 
ments are combined with the PIAFS devised as a trans- 
mission control method for transmitting conputer data 
through the radio channel of PHS having a bit rate of 32 
Kbps, audio data can be multiplexed and transmitted 
without jitter and delay while adopting the multiplex 
frame length providing high transmission efficiency 

Although the seventh to ninth embodiments have 
eixplained multiplexing and transmitting of the audio 
data and the video data, other data such as the compu- 
ter data may be transmitted as the variable-lengtti data 
enrploying retransmission, or the audio data, the video 
data and the computer data can be multiplexed and 
transmitted to provide the same effects. 

In the seventh to ninth embodiments, even when a 
plurality of audio frames are multiplexed in one multiplex 
frame, the audio jitter can be absorbed as in the embod- 
iments above by regarding the multiplex information of 
the multiplex frame as a reference multiplex frame and 
using the first audio frame as a reference audio frame. 
In this case. Step S202 in Fig.20 determines whether a 
plurality of audio frames are included In one multiplex 
frame and Step S206 regards ttie leading audio frame 
as a reference audio frame unconditionally. 

It depends on the processing capability on the 
receiving side and the multiplex structure to which of the 
timings in the multiplex structure of Fig. 13(d) the timing 
on the left end in Rg. 13(b) and (c) corresponds. It may 
be immediately after reception of an audio frame, or 
may be after completion of the reception of the multiplex 
frame. When the relation between the multiplex struc- 
ture and the multiplex information is defined by using 
the multiplex code table in Fig. 19 with one nujltiplex 
frame having one multiplex header, the timing will be 
like that shown in Fig.13(b), (c), (d). When the relation 
between the multiplex structure and the multiplex infor- 
mation is defined by using the multiplex code table of 
Fig.21, the timing will be like that shown in Fig.22(a), 
(b). (c). When the oon^espondence between the multi- 
plex structure and the multiplex information is defined 
by using the muttiplex code table shown in Fig. 19 with 
one multiplex frame provided with two multiplex headers 
at the head and the end, ttie timing will be like that 
shown in Fig.23(a}. (b), (c). As can be seen by refening 
to each of the diagrams, the present invention can simi- 
larly provide the effects in any of the multplex struc- 
tures. 

The eighth and ninth embodiments have shown 
examples in which no synchronization flag is provided at 
the head of the multiplex frame. However, the present 
invention can be applied to provide the effects shown in 
the embodiments as long as the audio frames are trans- 
mitted in multiplex frames with different cycle, even 
when the multiplex frames always have a synchroniza- 
tion flag at the head without utilizing synchronization by 
frames, or when the multiplex frames are defined by 



another format, or when the information to be multi- 
plexed includes data or control information. 

Although the eighth and nintfi embodiments have 
explained the case in which the audio frame has a 
5 length of 25 bytes, the present invention is not limited to 
it. That is to say, the frame length of the audio frame can 
take other values. Audio frames provided with redun- 
dant bits for error correction, or audio frames coded to 
another bit rate, e.g., 6.3 Kbps, may be used as well. 



10 



(Tenth Preferred Errbodiment) 



In the atK)ve-described seventh to ninth embodi- 
ments, it is necessary for the transmitting side to multi- 

15 plex audio codes generated at constant intervals in the 
audio fiekis of the multiplex frames having jitter with a 
minimum delay. That is to say, unlike the receiving side, 
the transmitting side must store the audio code into the 
audio frame field separated at a relatively short interval 

20 from the next audk) frame field with a delay equal to or 
larger than the jitter time, on the basis of the audio 
frame f ieki separated from the next audio frame at a rel- 
atively long interval. A large delay time causes no prob- 
lem when storing, but increases the delay time of audio 

25 reproduced on the receiving skJe. This brings about ttie 
necessity of preventing an increase of the delay time by 
making the relation between occurrence of the audio 
frames and the phase of the multiplex frames constant. 
For this aim, the transmitting terminal shall control the 

30 operation start timing of the audio coder. 

The time from the start of the audio coder to the 
output of the audio code is taken as T, the jitter time of 
the time intervals ttetween the audio frames in the mul- 
tiplex frames as J, and the time required for determining 

35 whether the audio frame is an active speech frame or a 
silent frame, assembling a multiplex frame correspond- 
ing to the determination, and sending out it onto tiie 
transmission channel as a. When the first multiplex 
frame is sent out onto tiie transmission channel immedi- 

40 ately after the total time (T+J-ha ) has passed after the 
start of the audio coder, tiie audio delay time can be 
minimized while absorbing the time interval jitter 
between tiie audio frames. Considering this, tiie tenth 
embodiment of the invention determines the most suita- 

45 ble start timing of the audio coder. 

Fig.24 is a tinriing chart showing the operation on 
the transmitting side in the multiplex transmission 
method according to ttie tenth embodiment of the 
present invention. In the tentii embodiment, tiie relation 

50 between the multiplex structure and the multiplex infor- 
mation is defined by using the multiplex code table 
shown in Fig. 14. 

Fig.24(a) shows the timing for starting the audio 
coder. In Fig.24(a), t is tiie time required from when the 

55 audio coder is started to when the audio code is output- 
ted. Fig.24(b) shows tiie timing of output of the audio 
code from ttie audio coder. Fig.24(c) shows ttie timing at 
which tiie multiplex frame is outputted onto the trans- 
mission channel. In the case of the multiplex frame 
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stream shown in Fig.24(d), the time intervals between 
the audio frames (repeating 80-byte intervals and 160- 
b/te intervals) cause 40-byte jitter with respect to the 
cycle (120-byte cycle) of occurrence of the audio codes. 
That is to say, J=40 bytes. The character a represents 5 
the time margin required to determine whether speech 
is active or inactive after the audio frame is outputted, 
assemble a multiplex frame corresponding to the deter- 
mination, and send out it onto the transmission channel. 
In this embodiment, a is set to a value larger than the 
longer one of the time margins for the speech period 
and the silent period. 

From the relation above, when the time at which the 
first multiplex frame goes out onto the transmission 
channel is used as a reference, this embodiment sets 
the start timing of the audio coder so that it starts pre- 
ceding the reference time for the total time (T-Ki+a). If 
the time difference between the start timing of the audio 
coder and the time at which the first multiplex frame is 
sent out onto the transmission channel is set to a value 
larger than the total time (T+J+a ), it causes no problem 
when multiplexing, but leads to an audio delay. Accord- 
ingly, it is desirable that the time difference is set as 
close to the total time (T-Kl+a) as possible. 

The tenth embodiment thus enables multiplexing 
and transmitting of the 120-byte cycle audio on the 80- 
byte multiplex frames without jitter nor delay. When the 
jitter absorbing method of this embodiment Is combined 
with PIAFS devised as a transmission control method 
for transmitting computer data on the radio channel of 
PHS having a bit rate of 32 Kbps. the audio data can be 
multiplexed and transmitted without jitter and delay 
while adopting the nnjitlplex frame length providing high 
transmission efficiency. 

When multiplex frames have the multiplex informa- 
tion (MC) from which a reference audio frame can be 
identified, as those in the eighth and ninth embodi- 
ments, the timing will be that shown in Fig.25. The rela- 
tion between the multiplex structure and the multiplex 
information in Fig.25 is defined by the multiplex code 
table shown in Fig.19. In Fig.25. MC=1 indicates a nnul- 
tiplex frame including audio data generated after being 
delayed for J time after the audio code was outputted, 
and MC=2 indicates a multiplex frame including audio 
data generated immediately after the audio frame was 
outputted. This enables the receiving side to Know 
whether the audio frame was stored after being delayed 
or stored at once to easily specify an audio frame as a 
reference for reproduction. When the relation between 
the multiplex structure and the multiplex information is 
defined by using the multiplex code table shown in 
Fig. 15, the timing is like that shown in Fig.26 when the 
video data and the audio data are multiplexed in this 
order. The timing of Fig.26 is fcased on the fact that it 
must be known whether the audio frame is speech- 
active or speech-Inactive at the time of determining the 
frame structure. In any cases, the present invention pro- 
vides the same effects as the tenth emtxxjiment. 

When a plurality of audio frames are multiplexed in 
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one multiplex frame with the number of the multiplexed 
frames being f. the invention provides the effects above 
with J=:120 xf bytes. 

Although the tenth embodiment has explained the 
case in which the audio frame has a length of 25 bytes, 
the present invention is not limited to It. That is to say, 
the frame length of the audio frame can take other val- 
ues. Audio frames provided with redundant bits for error 
correction, or audio frames coded to another bit rate, 
e.g.. 6.3 Kbps, may be used as well. 

(Eleventh Preferred Embodiment) 

Even when a complete ARQ frame is included in a 
fixed-length multiplex frame, if the frames are multi- 
plexed as shown in Fig.27. the position of the ARQ 
header of the video frame differs depending on whether 
an audio frame is multiplexed in the information field 
(Fig.27(a)) or no audio frame is multiplexed (Fig.27(b)). 
Accordingly, if the multiplex header has an error, the 
position of the ARQ header of the video frame is lost If 
the position of the ARQ header is not known, it is impos- 
sible to know the frame number of the transmission 
frame, the frame number of an error Kietected frame, the 
reject command, and so on. Then an error must be 
checked for from the next correctly an'ived frame and 
time-out in a time-consuming manner. 

Described next is a multiplex transmission method 
which enables analysis of the ARQ header of the video 
frame and an immediate retransmission request even if 
the multiplex header of the nxjltiplex frame has an emor. 

Rg.28 is a diagram showing multiplex structures of 
multiplex frames used in the multiplex transmission 
method according to the eleventh embodiment of the 
invention. As shown in Fig.28, In this err^odiment. the 
video frame is always multiplexed in the preceding posi- 
tion in the information field. Then the ARQ header of the 
video frame can be always located in the same position, 
irrespective of whether an audio frame is multiplexed in 
the infornrBtion field (Fig.28(a)) or no audio frame is 
multiplexed (Fig.28(b)). This allows the receiving side to 
know the position of the ARQ header of the video frame 
even if the multiplex header has an error. 

Fig.29 is a flow chart showing the processing per- 
formed by the multiplex layer on the receiving side. In 
fng.29, on receiving a multiplex frame, the multiplex 
layer on the receiving side first applies error detection to 
the multiplex header (Step SOI). If the multiplex header 
has no error, the multiplex layer analyzes the multiplex 
information block (Step S02). demultiplexes the infor- 
mation field into tiie video frame, the audio frame, or the 
like, according to the multiplex information, and gives 
them to tiie corresponding error control processing lay- 
ers (Step S03). On the other hand, when detecting an 
error in the multiplex header, the multiplex layer extracts 
the ARQ header of the video frame stored in the fixed 
position in the multiplex frame and gives it to the video 
error control processing layer (Step SIS). 

Fig.30 is a flow chart showing the processing per- 
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formed by the video error control processing layer. In 
Flg.30, the error control layer first determines whether it 
has received a demultiplexed video frame from the mul- 
tiplex layer (Step SI 4). 

When having received a demultiplexed video frame 
from the multiplex layer (that is, when the multiplex 
header has no en'or), the error control processing layer 
performs en'or correction and error detection of the 
ARQ header (Step SOS). If the ARQ header has no 
error, the error control processing layer reads the frame 
number (Step S06). Next, the error control processing 
layer determines whether a sequence error has been 
detected, or whether the read frame number differs from 
the frame number to be received next (Step SOT). When 
a sequence error is detected, the error control process- 
ing layer transmits a retransmission request for the 
video frame of the absent frame number to the transmit- 
ting side (Step S08). Then the error control processing 
layer performs error detection of the video packet (Step 
S09). If the video packet has no error, the eror control 
processing layer outputs the video data to the video 
decoding layer (Step S10). and if the video data has an 
error, the error control processing layer transmits a 
retransmission request for that frame nunrt}er to the 
transmitting side (Step S11). In the Step SOS, when an 
error is detected in the ARQ header, the error control 
processing layer discards the video packet (Step S12). 

On the other hand, when it has received only the 
ARQ header of the video frame from the multiplex layer 
in Step S14 (that is. when the multiplexing header has 
an error), the error control processing layer performs 
error correction and error detection of the received ARQ 
header (Step S15). When the ARQ header has no error, 
the error control processing layer reads the frame 
number (Step S16). Next, the error control processing 
layer determines whether a sequence error has been 
detected, or whether the read frame number differs from 
the frame number to be received next (Step SI 7). When 
a sequence error is detected, the error control process- 
ing layer transmits a retransmission request for the 
video frame of the absent frame nuni}er to the transmit- 
ting side (Step S18). Then the error control processing 
layer transmits a retransmission request for that read 
frame number to the transmitting side (Step SI 9). When 
an en'or is detected in the ARQ header in Step S15, the 
en-or control processing layer discards the video packet 
(Step S20). 

Fig.31 is a timing chart showing the operation when 
retransmission control is performed in the eleventh 
embodiment. R1 in Fig.31 indicates a reject transmitted 
back for the video 1. In ng.31, when a transmission 
error takes place in the multiplex header of the multiplex 
frame (the multiplex frame containing the video frame of 
the frame number 1) transmitted from the terminal 1 , the 
terminal 2 receives it and then analyzes the ARQ 
header of the video frame. If the ARQ header has no 
error, it sends a reject. Thus the terminal 2 can immedi- 
ately tramsmit back a reject for the frame number 1 . 

Thus, in the eleventh embodiment, a video frame is 



always multiplexed in the head part of the multiplex 
frame. Hence, even if the multiplex header has an error, 
the ARQ header of the video frame can be analyzed so 
that a reject for the video frame can immediately be 
5 transmitted back, and retransmission delay of the video 
frame are thus reduced. 

This embodiment provides more efficient effects 
when the ARQ headers are extracted only when all mul- 
tiplex frames contain video data multiplexed therein. 

10 

(Twelfth Preferred Embodiment) 

Next, a multiplex transmission method according to 
a twelfth embodiment of the present invention will be 
15 explained. This embodiment is characterized in that the 
multiplex layer demultiplexes a video frame of certain 
bytes and gives it to the enor control processing layer 
even if the multiplex header of the multiplex frame has 
an en-or. 

20 If the multiplex code table for the multiplex infomna- 
tion signaled from the transmitting side at the beginning 
of a communication is the multiplex code table shown in 
Fig. 19, the receiving side determines that the ARQ 
headers of the video frames are not located in a fixed 

25 position in the multiplex frames. In this case, the multi- 
plex layer on the receiving side performs the processing 
shown in Rg.32 and the video en-or control processing 
layer perfonns the processing shown in ng.33. 

In Fig.32, when receiving a multiplex frame, the 

30 multiplex layer on the receiving side first performs en^or 
detection of the multiplex header (Step S0 1). If the mul- 
tiplex header has no error, the multiplex layer analyzes 
the multiplex information block (Step S02), demulti- 
plexes the information fiekJ into the video frame, the 

35 audio frame, and so on. according to the multiplex infor- 
mation, and gives them to the corresponding en'or con- 
trol processing layers (Step SOS). On the other hand, 
when detecting an enor in the multiplex header, the 
multiplex layer discards tiie data in the information field 

40 since the multiplex information for the information field is 
unknown (Step S04). 

In Fig.33, when having received a demultiplexed 
video frame from the multiplex layer (that is, when the 
multiplex header has no error), the error control 

45 processing layer performs en-or con-ection and enror 
detection of the ARQ header (Step SOS). If the ARQ 
header has no error, the error control processing layer 
reads the frame number (Step S06). Next, the error con- 
trol processing layer determines whether a sequence 

so error has been detected, i.e., whether the read frame 
number differs from tiie frame nunrt)er to be received 
next (Step S07). When a sequence error is detected, 
the error control processing layer transmits a retrans- 
mission request for the video frame of the absent frame 

55 number to the transmitting side (Step S08). Then the 
error control processing layer performs error detection 
of the video packet (Step S09). If the video packet has 
no en'or, it outputs the video data to the video decoding 
layer (Step SIO), and if it has an error, it transmits a 
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retransmission request for that frame number to the 
transmitting side (Step Si 1). In the Step SOS, when an 
error Is detected in the ARQ header, the error control 
processing layer discards the video packet (Step S12). 

On the other hand, if the multiplex code table of the s 
multiplex information Is the multiplex code table shown 
in Fig.2l, the receiving side determines that the ARQ 
headers of the video frames are always located in a 
fixed position in the nxiltiplex frames. In this case, the 
multiplex layer on the receiving side performs the io 
processing shown in Fig.34 and the video erra control 
processing layer performs the processing in Rg.33. 

In Rg.34, when receiving a multiplex frame, the 
multiplex layer on the receiving side first performs error 
detection of the multiplex header (Step S01). If the mul- is 
tiplex header has no error, the multiplex layer analyzes 
the multiplex information block (Step S02), demulti- 
plexes the information field into the video frame, the 
audio frame, and so on, according to the multiplex infor- 
mation, and gives them to the corresponding error con- 20 
trol processing layers (Step S03). On the other hand, if 
an en'or is detected in the multiplex header, the multi- 
plex layer demultiplexes the 51 bytes following the mul- 
tiplex header as a video frame and gives it to the video 
error control processing layer, and demultiplexes the 25 2s 
bytes at the end as an audio frame and gives it to the 
audio error control processing layer (Step S21). 

In Fig.33. if the multiplex layer has erroneously 
demultiplexed the information field but the ARQ header 
has no en'or (Step SOS), the video en-or control process- 30 
ing layer can con-ectly identify the frame number (Step 
S06). Then it is possible to detect an error in the video 
packet (Step S09) and a retransmission request for that 
frame can be transmitted (Step S1 1). 

Similarly, the audio error control processing layer 35 
performs error detection of the audio packet to discard 
or make a retransmission of the audio packet, tf, by 
chance, the video frame and the audio frame demulti- 
plexed by the muttiplex layer are correct, no error is 
detected in the video packet and the audio packet and 40 
the data are handled as valid data. 

Thus, in the twelfth embodiment, even if the multi- 
plex header has an error, a video frame of certain bytes 
from the head and an audio frame of certain bytes from 
the end are demultiplexed and given to the error control 45 
processing layers. This allows the video and audio error 
control processing layers to immediately transmitted 
back a reject for the en-ored frame, reducing the retrans- 
mission delay 

This embodiment will more effectively provide the so 
effects when it is determined on the basis of the multi- 
plex code table for the multiplex structure whether the 
ARQ headers of the video frames are located in a fixed 
position in the multiplex frames and the ARQ headers 
are extracted only when they are located in the fixed ss 
position. 

This embodiment will provide more remarkable 
effects if the video frame and the audio frame are 
demultiplexed, when the ARQ header has an error, on 



the basis of a most frequently appearing one of the 
video and audio multiplex structures, or if they are 
demultiplexed by predicting the multiplex structure when 
the multiplex structures appear in a repeated certain 
pattern. 

Furthernrrare, when multiplexing computer data in 
addition to the video data and the audio data, multiplex- 
ing the computer data between the video data at the 
head and the audio data at the tail will allow the video 
data and the audio data to be extracted with priority, pro- 
viding more significant effects. 

Although the twelfth embodiment has explained the 
case in which the audio frame has a length of 25 bytes, 
the present invention is not limited to it. That is to say, 
the frame length of the audio frame can take other val- 
ues. Audio frames provided with redundant bits for error 
correction, or audio frames coded to another bit rate, 
e.g., to 6.3 Kbps, may be used as well. 

(Thirteenth Preferred Embodiment) 

Next, a multiplex transmission method having the 
features of the above-described embodiments will be 
explained. Fig.35 is a diagram for use in describing a 
multiplex transmission method according to a thirteenth 
embodiment of the invention. Specifically, Fig.35(a) 
shows a multiplex frame stream transmitted by tiie 
transmitting side, Fig.35(b) shows a multiplex frame 
stream received on the recaving side, and Fig.35(c) 
shows the timing of outputting audio on the receiving 
side. 

As shown in Rg.35(a). each multiplex frame in the 
multiplex frame stream has a 80-byte fixed length. In 
tills multiplex frame stream, the pattern repeats in which 
two multiplex frames with 20-byte audio frames multi- 
plexed therein continuously appear and then one multi- 
plex frame with no audio frame multiplexed therein 
appears. Thus 20-byte audio is transmitted per 120 
bytes. When an error exists in the video frame V5 which 
includes a long frame because of inactive speech, this 
video frame V5 can be retransmitted in a multiplex 
frame witii no audio frame multiplexed (which necessar- 
ily appears once or more in successive tiiree multiplex 
frames). 

In the data to be multiplexed, tiie video data is 
always stored in the leading part of the multiplex frame. 
Even if an en'or is detected in the multiplex information, 
tiie receiving side demultiplexes the leading part as a 
video frame. Then, if tiie error exists only In the multi- 
plex information, tiie demultiplexed video frame V4 can 
be cc»'rectiy processed. 

Multiplex information MC having different values 
are stored in the header parts depending on the time to 
ttie next audio frame. On the receiving side, an audio 
frame separated from tiie next audio frame In a rela- 
tively ^ort time on the multiplex frame stream is imme- 
diately outputted witiiout being accumulated and tiie 
following audio frames are outputted on the 1 20-byte 
time cycle. This enables multiplexing of audio of 120- 
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byte cyde in 80-byte multiplex frames without jitter and 
delay. 

Fig.36 shows a method of multiplex transmission 
with each multiplex frame having a length of 60 bytes. 
Particularly, Fig.36(a) shows a multiplex frame stream 
transmitted from the transmitting side. Fig.36(b) shows 
a multiplex frame stream received on the rec^ving side, 
and Fig.36(c) shows the timing at which the receiving 
side outputs audio. 

As shown in Fig.36(a), each multiplex frame on the 
multiplex frame stream has a fixed-length of 60 bytes. 
On this multiplex frame stream, a multiplex frame con- 
taining a 20-byte audio frame and a multiplex frame 
without an audio frame alternately repeat. This enables 
transmission of 20-byte audio per 120 bytes. When the 
video frame V3 which includes a long frame due to 
silence has an error, this video frame V3 can be retrans- 
mitted in a multiplex frame containing no multiplexed 
audio frame (which necessarily appears once per suc- 
cessive two multiplex frames). 

In the data to be multiplexed, the video data is 
always stored In the preceding part of the nnultiplex 
frame. Even if an error is detected in the mult^itex infor- 
mation, the receiving side demultiplexes the head part 
as a video frame. This allows the demultiplexed video 
frame V5 to be correctly processed if only the nnultiplex 
information has an en^or. 

Thus, according to the thirteenth embodiment, the 
data link frames for video data can always be obtained 
with constant timing when audio frames generated on 
the 1 20-byte cycle are multiplexed in 80-byte multiplex 
frames or when multiplexed In 60-byte nnultiplex frames. 
Accordingly, even when the silent periods are effectively 
used by increasing/decreasing the transmission band 
width for video data or computer data, or the variable- 
length data employing retransmission, the transmitting 
timing of the data link frames can be synchronized with 
the transmitting timing of the multiplex frames, resulting 
in a multiplex transmission with improved en'or robust- 
ness. Furthermore, retransmission of a long frame of 
the variable-length slot 2 can be perfomned by using the 
fixed-length slot, which enables retransmission control 
without delay of audio. 

Although the thirteenh ennbodiment has explained 
the case in which the audio frame has a length of 20 
bytes, the present invention is not limited to it. Tliat is to 
say the frame length of the audio frame nnay be other- 
wise determined. Audio frames provided with redundant 
bits for error correction, or audio frames coded to 
another bit rate, e.g.. to 6.3 Kbps. may be used as well. 

(Fourteenth Preferred Embodiment) 

Consider that, in a communication system in which 
a PHS network is connected to an existing analogue 
public switched telephone network PSTN as shown in 
Fig.37, video data and audio data are multiplexed from 
a terminal 2101 connected to the PHS nehwork to a ter- 
minal 2103 connected to the PSTN. 



It is assumed that the video data retransmission 
control is made between the terminal 2101 and the ter- 
minal 2103. The two terminals determine the maximum 
bit rate between the terminals according to the definition 

5 of H.324 by ITU-T at the beginning of a communication, 
for example. Since the PHS network has a bit rate of 32 
Kbps and the PSTN has a bit rate of 28.8 Kbps, the 
maximum bit rate of this communication system is 
determined to be 28.8 Kbps. 

10 At the interconnection of the PHS network and the 
PSTN network, the video data is the data employing 
retransmission and is retransmitted when enroneously 
received. Video data encored in the PHS network is dis- 
carded at the network connection 2102 and a retrans- 

75 mission request is issued. Accordingly, if an en^or occurs 
in the video data In the PHS network, the less video 
data to be multiplexed is in the PSTN network, the more 
is retransmitted, and then the PSTN network actually 
has less data to be transmitted than 28.8 Kbps. That is 

20 to say, the effective bit rate falls under 28.8 Kbps. As 
compared with the wired transmission path of PSTN, 
the radio transmission channel of PHS is more prone to 
errors and retransmission is made nnore frequently 
Accordingly, the bit rate between the terminals is limited 

25 by the bit rate in the PHS network, leading to inferior 
transmission efficiency 

The fourteenth embodimerrt described next is 
directed to a solution of tiie deterioration of the trans- 
mission efficiency 

30 The multiplexing method at the terminal 2101 will 
now be described referring to Fig.38. The terminal 2101 
cuts out 308 bits from the bit stream of video data (refer 
to Fig.38(a)) coded by H.263 or the like and adds a 16- 
bit CRC and 4-btt termination bits to it as shown in 

35 Fig.38 (b). The CRC is the code for en-or detection and 
the termination bits are dummy bits for Viterbi decoding. 
The temiinal 2101 applies enror correction coding with a 
code rate r=8/9 by rate compatible punctured convolu- 
ttonal coding (RCPC coding) to the total of 328^Dit (=41- 

40 b»yte) data to generate an RCPC packet of 328 bits x 
(9/8) = 369 bits. The terminal 2101 adds the 32-bit ARQ 
header including information such as the sequence 
number for retransmission control to the head of this 
RCPC packet to generate the 401 -bit (^53 bytes) video 

45 data shown in Fig.38(c). Further, the terminal 2101 mul- 
tiplexes the video data and the audio data to generate 
the 80-byte multiplex frame shown in Fig.38(d). Then 
the terminal 2101 generates this 80-byte nrultiplex 
frame once per 80 bytes/32 Kbps = 20 msec. 

50 Next, the rate converting method at the network 
connection 2102 will be explained referring to Fig.39. 
The network connection 2102 releases the multiplexing 
of the multiplex frame shown in Fig.39(a) and demulti- 
plexes the video data into the ARQ header and the 

55 RCPC packet, as shown in Fig.39(b). Next, the network 
connection 2102 applies Viterbi decoding (error correc- 
tion) to the RCPC packet to take out 41 -byte data shown 
in Fig.39(c). Next, the network connection 2102 applies 
en-or correction coding with a code rate r=8/8=1 to the 
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41 -byte data by RCPC coding to generate a 41 -byte 
RCPC packet. Furthermore, the network connection 
2102 adds, as shown in Fig.39(d), the previously demul- 
tiplexed 32-bit ARQ header to the head of the RCPC 
packet to generate 45-byte video data. Next, the net- 
work connection 2102 multiplexes the 45-byte video 
data with the previously demultiplexed 21 -byte audio 
data to generate the 72-byte multiplex frame shown in 
Fig.39(e) and transmits it on the PSTN network. The 72- 
byte multiplex frame is generated once per 20 msec, 
then 72 bytes/20 msec = 28.8 Kbps. which can be trans- 
mitted on the PSTN network. 

The code rate Is determined as follows. The PHS 
network has a bit rate Vf of 32 Kbps. and the PSTN has 
a bit rate Vs of 28.8 Kbps. Therefore the PHS network 
has a bit rate higher than that of the PSTN by 

Rv = (32 -28.8) 732 = 0.1 

that is. by 10%. Hence, 80 x 0.1 = 8 bytes in the frame 
length Lm (=80 bytes) of the multiplexed frame in the 
PHS network is empty in a conventional manner and the 
frame length of the multiplex frames in PSTN is there- 
fore 80 - 8 = 72 bytes. The length UJ of the video frame 
in the multiplex frame in PSTN corresponds to the mul- 
tiplex frame length minus the header, the audio data, 
and the ARQ header of the video frame, that is. 

Ld = 72-2-4-21-4 = 41 bytes. 

The 8 bytes are assigned to the redundant bits for error 
correction of the video frame, and then the code rate Rc 
= 41/(41 + 8 ) = 41/49 or larger. Preferably, selecting a 
value close to this value (=41/49) provides the best effi- 
ciency In the RCPC coding explained at>ove, the code 
rate must be 8/n (8^n^32). Hence, considering that the 
condition 41/49 ^ 8/9 holds, this embodiment selects 
the code rate of 8/9. 

Although the fourteenth embodiment performs error 
correction of video data by using oonvolutional coding, 
block coding may be used. In this case, the network 
connection 2102 can make the rate conversion just by 
discarding the parity bits without performing error cor- 
rection. 

In the fourteenth embodiment, the error con-ection 
coding Is applied to the video data with a retransmission 
processing. Similarly, the error correction coding may 
be applied to the audio data without the retransmission 
processing and the netwak connection 2102 may con- 
vert the code rate of the error correction coding. 

In the fourteenth embodiment, synchronization 
flags are provided at the heads of the multiplex frames. 
However, as explained in other embodiments described 
above, the synchronization flags are not essential. 

The PHS network and the PSTN network in the 
fourteenth emtxKliment may be a customer-provided 
PHS network and a customer-provided analogue tele- 
phone network. 

As has been explained above, the fourteenth 
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embodiment takes advantage of the fact that the PHS 
network has a bit rate 10% higher than that of the PSTN 
to increase the error correcting capability of the video 
data. This increases the effective bit rate in the error- 
5 prone PHS network, resulting in suppressing reduction 
of the transmission efficiency between terminals. 

(Fifteenth Preferred Embodiment) 

10 Described next is a method for multiplexing and 
transmitting video data and audio data from the terminal 
2101 connected to the PHS network to the terminal 
2103 connected to the PSTN in such a communication 
system as shown in Fig.37 wherein the PHS network is 

15 connected to the existing analogue public switched tele- 
phone network PSTN, tike that in the fourteenth embod- 
iment. 

In this embodiment, retransmission control of video 
data Is performed between the terminal 2101 and the 

20 network connection 2 1 02 and between the network con- 
nection 2102 and the terminal 2103. The PHS network 
has a bit rate of 32 Kbps. and the PSTN has a bit rate of 
28.8 Kbps, and the network connection 2102 performs 
conversion of the bit rate and conversion of multiplexing. 

25 The terminal 2101 multiplexes and transmits data 
cut out from the video bit stream coded by H.263 or the 
like and audio data at 32 Kbps. 

The structure of the network connection 2102 in 
this embodiment is shown in Fig.40. In Fig. 40, the net- 
so work connection 2102 includes a first multiplexing block 
2401. a second multiplexing block 2402 and a video 
buffer 2406. The first multiplexing block 2401 includes a 
multiplex layer 2403. a video error control block 2404, 
and an audio discarding block 2405. 

35 The multiplex layer 2403 in the first multiplexing 
block 2401 releases the multiplexing of a multiplex 
frame received from the PHS network and demulti- 
plexes the multiplex frame into video data and audio 
data. The video error control block 2404 performs error 

40 correction or error detection of the video data and per- 
fbmis retransmission control of en-or-uncorrected or 
error-detected video data. The video error control block 
2404 rean-anges the video data subjected to the 
retransmission control In the order of the sequence 

45 numbers to generate a bit stream having no error and 
stores it in the video buffer 2406. The audio discarding 
block 2405 perfornDs error detection of the audio data, 
and discards error-detected audio data and outputs 
error-free audio data only to the second multiplexing 

50 block 2402. The second multiplexing block 2402 multi- 
plexes the bit stream of the error-free video data stored 
in the video buffer 2406 and the error-free audio data at 
28.8 Kbps. If an error occurs in the audio data and the 
aidio data is d^carded, the second multiplexing block 

55 2402 assigns that band to the video data in multiplexing. 
When the effective bit rate In the PHS network is 
larger than that in the PSTN networK data is accumu- 
lated in the video buffer 2406 to cause an overflow. 
When the video buffer 2406 overflows, the video 
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retransmission control block 2404 sends a retransmis- 
sion request for the data which could not be stored into 
the buffer to the transmitting terminal 2101. Therefore, 
actually, the effective bit rate In the PHS network does 
not exceed the effective bit rate in the PSTN network. 

Accordingly, according to this emtxxjiment, the bit 
rate between the two terminals is equal to the effective 
bit rate in the PSTN network unless the effective bit rate 
in the PHS network falls under the effective bit rate in 
the PSTN network This prevents a decrease of the 
transmission efficiency. The transmission efficiency can 
be further increased by assigning the field for the 
errored audio data to the video data. 

In this embodiment, synchronization flags are pro- 
vided at the heads of the multiplex frames. However, as 
explained in other embodiments above, the synchroni- 
zation flags are not essential. 

The PHS network and the PSTN network in this 
embodiment may be a customer-provided PHS network 
and a customer-provided analogue telephone network. 

(Sixteenth Preferred Embodiment) 

Considered next is a communication in such a com- 
munication system as shown in Fig.41 wherein a PHS 
network is connected to an existing analogue public 
switched telephone network PSTN, in which a terminal 
2601 connected to the PHS network communicates with 
a terminal 2602 connected to the PHS network and a 
terminal 2604 connected to the PSTN. 

In Fig.41, the interconnecting device 2603 has a 
function of PHS bearer communication, a function of 
communication in a PSTN analogue telephone network 
through a modem, a function of acquiring frame syn- 
chronization of the muttiplex frames in the individual net- 
works, and a function of rate conversion. 

Fig.42(a) shows a structure of a multiplex frame 
transmitted by the terminal 2601 connected to the PHS 
network The multiplex frame has the same frame 
length of 80 bytes as the data frame in the PIAFS and 
contains "01010000111011110010100110010011" as 
a synchronization flag in the head part of the frame, for 
example. This is followed by a header including the mul- 
tiplex information, and multiplexed audio data and video 
data, and then 8-byte dumnry bits are filled. The termi- 
nal 2602 and the interconnecting device 2603 establish 
frame synchronization by using the PIAFS synchroniza- 
tion frame since the multiplex frames has the same 
frame length as the PIAFS data frame. 

The interconnecting device 2603 eliminates the fill 
bits to convert the multiplex frame shown in Flg.42(a) 
Into the multiplex frame shown in Rg.42(b) to convert 
the bit rate. 

The terminal 2604 receives the multiplex frames 
while keeping frame synchronization by using the syn- 
chronization flags at the heads of the multiplex frames 
of Fig.42{b). 

According to this embodiment, the terminal 2602 
and the interconnecting device 2603 do not require the 
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function of searching in the synchronization flags at the 
heads of tiie multiplex frames, and the interconnecting 
device 2603 Is not required to newly add the synchroni- 
zation flags. 

As methods for the bit rate conversion in the inter- 
connecting device 2603, the method shown in the four- 
teenth embodiment is also applicable as well as the 
method of eliminating previously inserted fill bits. 

This embodiment can be applied to the multiplex 
transmission methods explained in tiie first to fifth, tenth 
and eleventh embodiments to provide tiie effects of both 
of the embodiments. 

TTie PHS network and the PSTN network in tills 
embodiment may be a customer-provided PHS network 
or a customer-provided analogue telephone network, 
respectively 

When communicating witfi a terminal which is con- 
nected to the PHS network or to an ISDN network con- 
nected thereto but may be supposed to be connected to 
a network incapable of synchronization, the nruttiplex 
frames after the synchronization has been established 
by the synchronization frame may be transmitted with a 
synchronization flag at each head and then it will be 
negotiated whether to remove the synchronization flag. 

(Seventeenth Preferred En^odiment) 

Considered next is another embodiment of the 
communication in such a conrtmunication system as 
shown in Fig.41 wherein a PHS network is connected to 
an existing analogue public switched telephone network 
PSTN, in which the terminal 2601 connected to the PHS 
network communicates with the terminal 2602 con- 
nected to tiie PHS network and the terminal 2604 con- 
nected to the PSTN. 

In Fig.41, the interconnecting device 2603 has a 
function of PHS bearer communication, a function of 
communication in a PSTN einalogue telephone network 
through a modem, a function of acquiring frame syn- 
chronization of the multiplex frames in the individual net- 
works, and a function of tiie bit rate conversion. 

Fig.43 shows a structure of a multiplex frame trans- 
mitted by the terminal 2601 connected to the PHS net- 
work. As shown in Fig.43, as in the sixteenth 
embodiment, the multiplex frame has the same frame 
length of 80 bytes as the PIAFS data frame. Accord- 
ingly, tiie terminal 2602 and tiie interconnecting device 
2603 establish frame synchronization by using the 
PIAFS synchronization frame. When the other terminal 
in a communication is connected to the PSTN, the ter- 
minal 2601 uses, as shown in Fig.43(b), a multiplex 
frame having a synchronization flag stored at its head, 
as in the sixteenth embodiment. If the other terminal is 
connected to the PHS network, it uses a multiplex frame 
having no synchronization flag at its head, as shown in 
Fig.43(c). In this case, fill bits are not inserted into the 
multiplex frame since the bit rate conversion is not 
required. 

According to this embodiment unnecessary syn- 
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chronization flags are not provided when the other party 
is connected to the PHS network so that more data can 
be multiplexed. This improves the transmission effi- 
ciency. 

This embodiment can be applied to the multiplex 
transmission methods explained in the first to fifth, tenth 
and eleventh embodiments to provide the effects of both 
of the embodiments. 

TTie PHS network and the PSTN network in this 
embodiment may be a customerisrovided PHS network 
or a customer-provided analogue telephone network, 
respectively. 

It can be negotiated whether to insert synchroniza- 
tion flags between terminals at the beginning of a com- 
munication, or it may be estimated from the maximum 
bit rate negotiated bertween the terminals as explained 
in the fourteenth and fifteenth embodiments, or informa- 
tion may be exchanged to show the types of the net- 
works to which the terminals are connected. 

When communicating with a terminal which is con- 
nected to the PHS network or to an ISDN network con- 
nected thereto but may be supposed to be connected to 
a network incapable of synchronization, the multiplex 
frame after establishment of the synchronization frame 
may be transmitted with a synchronization flag at its 
head and then it may be negotiated as to whether to 
remove the synchronization flag. 

The multiplex transmission system, the multiplex 
transmission device and the jitter absorbing system 
desaibed in the above first to seventeenth embodi- 
ments may be realized by each dedicated circuit device 
or a computer device. When they are realized by the 
computer device, software program for storing the pro- 
cedure of processing in the above each emtxxjiment is 
executed by the connputer device. The present invention 
is further directed to a storage medium for storing soft- 
ware program, such as a flexible disk, CD-ROM, a hard 
disk device, a semiconductor memory and so on. 

While the invention has been described In detail, 
the foregoing description is in all aspects illustrative and 
not restrictive. It Is understood that numerous other 
modifications and variations can be devised without 
departing from the scope of the invention. 

Claims 

1 . A method for multiplexing and transmitting variable- 
length data employing retransmission to which 
en'or connection by retransmission control is applied 
and fixed-length data not employing retransmission 
to which error correction by retransmission control 
is not applied in multiplex frames. 

wherein sakl multiplex frames have a fixed 
length and a data link frame for storing said varia- 
ble-length data employing retransmission has a 
frame length increased/decreased depending on 
the length of said fixed-length data not employing 
retransmission to be multiplexed so that the data 
link frame is transmitted at timing kept in synchroni- 



zation, for each said multiplex frame, with the trans- 
mission stream of said multiplex frames. 

2. The multiplex transmission method according to 
5 claim 1, wherein said multiplex frame includes a 

fixed-length field, 

said fixed-length field being divided into first 
and second variable-length slots. 

said first variable-length slot having a prede- 
10 termined fixed length when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a length of zero when no fixed-length 
data not errploying retransmission to be multi- 
plexed exists. 

15 said second variatrie-length slot having a rel- 

atively short, first length when fixed -length data not 
employing retransmission to be multiplexed exists 
and having a relatively long, second length when no 
fixed-length data not employing retransmission to 

so be multiplexed exists, 

wherein said fixed-length data not employing 
retransmission is transmitted in said first variable- 
length slot and said variable-length data employing 
retransmission is transmitted in said second varia- 

25 ble-!ength slot. 

3. The multiplex transmission method according to 
claim 2, wherein said multiplex frame further 
includes a multiplex frame header, 

30 said multiplex frame header containing mul- 

tiplex information indicating whether said fixed- 
length data not employing retransmission is multi- 
plexed in said multiplex frame. 

35 4, The multiplex transmission method according to 
claim 2, wherein said multiplex frame further 
Includes a fixed-length slot for storing and transmit- 
ting said variable-length data employing retrans- 
mission. 

40 

5. The multiplex transmission method according to 
claim 4, wherein variable-length data employing 
retransmission of the same kind as the variable- 
length data employing retransmission stored in said 

45 second variable-length slot is stored in said fixed- 
length slot. 

6. The multiplex transmission method according to 
claim 5, wherein the length of said fixed-length slot 

50 is selected equal to or larger than the I ength of said 
fixed-length field. 

7. The multiplex transmission method according to 
claim 4, wherein variable-length data employing 

55 retransmission of a different kind from the variatrfe- 
length data employing retransmission stored in said 
second variable-length slot is stored in said fixed- 
length slot. 
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8. The multiplex transmission method according to 
claim 4, wherein the length of the data link frame for 
storing said variable-length data employing retrans- 
mission is set equal to that of said second variable- 
length slot when said variable-length data enriplo/- 5 
ing retransmission is transmitted in said second 
variable-length slot and equal to that of said fixed- 
length slot when said variable-length data emplp/- 
ing retransmission is transmitted in said fixed- 
length slot. 

9. The multiplex transmission method according to 
claim 1, wherein said multiplex frame includes, 



10. The multiplex transmission method according to 4S 
claim 9, wherein wh^ fixed-length data not 
employing retransmission to be multiplexed exists, 
said user data field is divided into said field for vari- 
able-length data employing retransmission and said 
field for fixed-length data not employing retransmis- so 
sion, said field for variable-length data employing 
retransmission storing said variable-length data 
employing retransmission and said field for fixed- 
length data rxrt enploying retransmission storing 
said fixed-length data not employing retransmis- ss 
sion, and 

when no fixed-length data not employing 
retransmission to be multiplexed exists, said varia- 
ble-length data employing retransmission is stored 



66 

by using the entirety of said user data field as said 
field for variable-length data employing retransmis- 
sion, 

wherein an identification code indicating 
whether said user data field includes said field for 
fixed-length data not employing retransmission is 
stored in said frame type identification field. 

11. The multiplex transmission method according to 
claim 1, wherein when fixed-length data not 
employing retransmission to be multiplexed exists, 
a basic ntultiplex frame stream including at least 
one first multiplex frame in which said fixed-length 
data not employing retransmission is multiplexed 
and at least one second multiplex frame in which 
said fixed-length data not employing retransmission 
is not multiplexed is repeatedly transmitted. 

12. The multiplex transmission method according to 
claim 1 1 , wherein said first multiplex frame includes 
a fixed-length field divided Into first and second var- 
iable-length slots, 

said first variable-length slot having a prede- 
termined fixed length when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a length of zero when no fixed-length 
data not employing retransmission to be multi- 
plexed exists. 

said second variable-length slot having a rel- 
atively short, first length when fixed-length data not 
employing retransnnission to be multiplexed exists 
and having a relatively long, second length when no 
fixed-length data not employing retransmission to 
be multiplexed exists, and 

said second multiplex frame includes a fixed- 
length slot, 

wherein said fixed-length data not employing 
retransmission is transmitted in said first variable- 
length slot and said variable-length data ennploying 
retransmission Is transmitted in said second varia- 
ble-length slot and said fixed-length slot. 

13. The multiplex transmission method according to 
daim 1 1 , wherein said first and second fixed-length 
nultiplex frames each have a length of 60 bytes, 
and 

said basic multiplex frame ^ream is formed 
of one of said first nrtultiplex frame and one of said 
second fixed-length multiplex frame. 

wherein one audio frame is stored as the 
fixed-length data not employing retransmission in 
said first nxiltiptex frame. 

14. The multiplex transmission method according to 
claim 1 1 , wherein said first and second fixed-length 
multiplex frames each have a length of 80 bytes, 
and 

said basic multiplex frame stream is formed 
of two of said first multiplex frames and one of said 
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a frame type identification field for storing a is 
code for identifying the frame type, 
a user data field for storing user data, 
a user data length indication field for storing the 
length of significant data in said user data field, 
a retransmission control field for storing a 20 
frame number used in retransmission control, 
and 

an error detecting code field for storing an error 
detecting code, 

wherein said user data field is divided ss 
into a field for the variable-length data enploy- 
ing retransmission and a field for the fixed- 
length data not employing retransmission, 
said variat)le-length data employing retrans- 
mission is stored in said field for variable-length 30 
data employing retransmission, 
said fixed-length data not employing retrans- 
mission is stored in said field for fixed-length 
data not employing retransmission, 
the length of said variable-length data employ- 35 
ing retransmission aoconnodated in said field 
for variable-length data employing retransmis- 
sion Is stored in said user data length indication 
field, and 

an error detecting code for enror detection in 40 
the part excluding said field for the fixed-length 
data not employing retransmission is stored in 
said en-or detecting code field. 
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second fixed-length multiplex frame, 

wherein said first multiplex frame contains 
one audio frame as said fixed-length data not 
employing retransmission. 

5 

15. The multiplex transmission method according to 
claim 1 1 , wherein said first ard second fixed-length 
multiplex frames each have a length of 80 bytes, 
and 

said basic multiplex frame stream is formed io 
of one of said first multiplex frame and two of said 
second fixed-length nujltiplex frames, 

wherein two audio frames are stored as said 
fixed-length data not employing retransmission in 
said first muttiplex frame. is 

16. A method of absorbing audio jitter on the side 
receiving multiplex frames when a low-bitrate- 
coded N-byte (N is a positive integer) audio frame 
generated at a rate of one frame per an F-byte (F is 20 
a positive integer and N<F) transmission time is 
stored and multiplexed in an M-byte (M ts a positive 
integer and M<F) fixed-length multiplex frame and 
when at most J-byte (J is a positive integer) audio 
jitter is generated among the audio fields of the mul- 2s 
tipiex frames because F/M is not an integer, 

wherein a reference audio frame separated 
from the next audio frame at a minimum interval is 
identified from among the audio frames on a stream 
of the transmitted multiplex frame, and 30 

using the timing at which said reference 
audio frame is outputted as a reference, the follow- 
ing audio frames are outputted at the rate of one 
frame per the F-byte transmission time. 

35 

17. The audio jitter absorbing method according to 
claim 1 6, wherein different multiplex information are 
defined for n multiplex frames containing multi- 
plexed audio frames in a basic multiplex frame 
stream fbnned of m multiplex frames, where n and 4o 
m are integers and n * F=m • M , so that said ref- 
erence audio frame can be identified on the basis of 
said multiplex information. 



jitter is generated among the audio fields of the mul- 
tiplex frames because F/M is not an integer, 

wherein when the time from the start of a 
coding operation of an audio coder for coding audio 
data to the start of output of the first audio frame 
from the audio coder is taken as T. and the time 
required from the start of output of the first audio 
frame from said audio coder to the start of output of 
the multiplex frame containing said first audio frame 
is taken as a, 

using as a reference the timing of the start of 
output of the nfujttiplex frame containing a reference 
audio frame separated from the next audio frame at 
a minimum interval in the audio frames on the 
transmitted multiplex frame stream, the coding^ 
operation of said audio coder is started preceding 
the reference timing at least for the total time of 
(T+J+a). 

21. The audio jitter absorbing method according to 
claim 20. wherein said time a is set equal to or 
larger than a larger one of first and second times, 

wherein said first time is a time required from 
the start of output of the audio frame from said 
audio coder to the start of output of the multiplex 
frame containing the audio frame, and 

said second time is a time required from the 
detection of the indication of silence outputted from 
said audio coder to the start of output of the multi- 
plex frame containing another data frame in the 
field to be stored In an audio frame. 

22. The audio jitter absortung method according to 
claim 20, wherein different multiplex Information are 
defined for n multiplex frames containing multi- 
plexed audio frames In a basic multiplex frame 
stream formed of m multiplex frames, where n and 
m are Integers and n * F=m * M , so that said ref- 
erence audio frame can be identified on the basis of 
said multiplex Information. 

23^ The audio jitter absorbing method according to 
claim 20, wherein M=^0 bytes and F=120 bytes. 



18. The audio jitter absorbing method according to 
claim 16, wherein M«80 bytes and F<=120 bytes. 

19. The audio jitter absorbing method according to 
claim 17, wherein M=80 bytes, F=120 bytes. n=2 
andm=3. 

20. A method of absorbing audio jitter on the side 
receiving multiplex frames when a low-bitrate- 
coded N-byte (N is a positive integer) audio frame 
generated at a rate of one frame per an F-byte (F is 
a positive integer and N<F) transmission time is 
stored and multiplexed in an M-byte (M is a positive 
integer and M<F) fixed-length multiplex frame and 
when at most J-byte (J is a positive Integer) audio 



45 24. The audio jitter absorbing method according to 
claim 22, wherein M=80 bytes, F=120 bytes, n=2 
and nr^=3. 

25. A method for multiplexing and transmitting a first 
50 data frame provided with a frame number and an 
error detecting code and second to ktii data 
frame(s) (k is an Integer of 2 or larger) In multiplex 
frames from a transmitting side to a receiving side, 
wherein said multiplex frame is further pro- 
55 vided with nruiltiplex Information I ndicating the struc- 
ture of the multiplexing of said first data frame and 
said second to Kth data frame(s), 

said transmitting side locates the frame 
number of said first data frame always in a fixed 
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position in said multiplex frame, and 

when detecting presence of an error in said 
multiplex information in the multiplex frame 
received, said receiving side reads the frame 
number of said first data frame from the fixed posi- 
tion in the multiplex frame and transmits a retrans- 
mission request for the read frame number to said 
transmitting side. 

26. The multiplex transmission method according to 
claim 25. wherein a fixed-length multiplex header 
including at least said multiplex information and an 
en-or detecting code for detecting an error in the 
multiplex information is located at least at the head 
of said multiplex frame, and 

said first data frame is always located adja- 
cent to said multiplex header among one or more 
data frames stored following said multiplex header. 

27. The multiplex transmission method according to 
claim 26. wherein said first data frame has a header 
including at least said frame number and an error 
detecting code for detecting an error of the frame 
number, 

wherein said header is located at least at the 
head when said first data frame is located adja- 
cently after said multiplex header and at least at the 
end when said first data frame is located adjacently 
before said multiplex header. 

28. The multiplex transmission method according to 
claim 27, wherein when detecting presence of an 
error in the multiplex header of said multiplex frame 
received, said receiving side extracts certain bytes 
of data following/preceding said multiplex header 
as said first data frame, and 

when detecting absence of en*or in the 
header of said extracted first data frame and detect- 
ing presence of an error in the remaining data, said 
receiving side immediately transmits a retransmis- 
sion request for the first data frame to said transmit- 
ting side. 

29. The multiplex transmission metiiod according to 
claim 26. wherein said multiplex frame has a fixed 
lengtii. 

said second data frame is always located in 
the first one or the last one of one or more data 
frames stored following said multiplex header, and 

when detecting presence of an error in the 
multiplex header of said multiplex frame received, 
and when said second data frame is located always 
in the last one, said receiving side extracts data in a 
range of certain bytes from the end of the informa- 
tion field of tiie multiplex frame as said second data 
frame, 

when said second data frame is always 
located in the first one, said receiving side extracts 
data in a range of certain bytes from the head of the 



information field of the multiplex frame as said sec- 
ond data frame, 

wherein when detecting absence of an error 
in said extracted second data frame, said receiving 
s side handles the second data frame as correctiy 
received data, and 

when detecting presence of an error in said 
extracted second data frame, said receiving side 
discards the second data frame. 

10 

30. A metiiod for multiplexing and transmitting a first 
data frame provided with a frame number and an 
error detecting code and second to kth data 
frame{s) (k is an integer of 2 or larger) in multiplex 

15 frames from a transmitting side to a receiving side, 
wherein said multiplex frame is further pro- 
vided witii multiplex information indicating in what 
structure said first data frame and said second to 
ktii data frame(s) are multiplexed, 

20 wherein said transmitting side includes as 

transmission modes. 

a first transmission nx)de in which the frame 
number of said first data frame is always located in 
a fixed position in said multiplex frame, and 

25 a second transmission mode in which tiie 

frame number of said first data frame is located in 
an arbitrary position in said multiplex frame. 

wherein said receiving side, when detecting 
presence of an error in said multiplex Information in 

30 the multiplex frame received in said first transmis- 
sion mode, reads tiie frame number of said first 
data frame from the fixed position in the nrultiplex 
frame and transmits a retransmission request for 
the read frame numt)er to said transmitting side. 

35 

31. The multiplex transmission method according to 
daim 30, wherein said transmitting side transmits to 
said receiving side, in advance, a multiplex code 
table indicating the correspondence between said 

40 multiplex infornnation and the multiplex structure of 

said multiplex frames, and 

said receiving side determines in which of 

said first and second transmission modes said 

transmitting side transmits the multiplex frames on 
45 the basis of said multiplex code table received in 

advance. 

32. A multiplex transmissfon method in which an audio 
frame having a frame lengtii of 30 ms and being of 

50 120-byte cycle when transmitted at 32 Kbps is 
stored, multiplexed and transmitted to a multiplex 
frame having a frame lengtii of 20 ms and being of 
80-byte cycle when transmitted at 32 Kbps, 

wherein a basic multiplex frame stream 

55 formed of two first multiplex frames containing said 
audio frame and one second multiplex fiame con- 
taining none of said audio frame is repeatedly 
transmitted. 
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33. The multiplex transmission method according to 
claim 32, wherein when an audio frame to be multi- 
plexed exists, said audio frame is always stored in 
the same position in an information field in said first 
multiplex frame, and 

a receiving side demultiplexes said audio 
frame from a multiplex frame received, absorbs jit- 
ter of the demultiplexed audio frame, and outputs 
the audio frame. 

34. The multiplex transmission method according to 
claim 33, wherein different multiplex information are 
defined for said two first multiplex frames included 
in said basic multiplex frame stream so that a refer- 
ence audio frame demultiplexed from the next audio 
frame at a minimum interval can be identified, and 

the receiving side outputs the audio frame 
demultiplexed from the received multiplex frame for 
each 30 ms using the timing of output of said refer* 
ence audio frame as a reference. 

35. The multiplex transmission method according to 
claim 32, wherein said first multiplex frame contains 
a first video frame which is one complete video 
ARQ frame, together with said audio frame, and 

said second multiplex frame contains a sec- 
ond video frame which is one complete video ARQ 
frame having a frame length equal to or larger than 
the sum of the frame length of said audio frame and 
that of said first video frame in said first multiplex 
frame. 

36. The multiplex transmission method according to 
claim 35, wherein when said audio frame is not 
transmitted in a silent period, said first muKiplex 
frame is used as a third multiplex frame which forms 
a third video frame which is one complete video 
ARQ frame having a frame length equal to the sum 
of the frame lengths of said audio frame and said 
first video frame, and 

when said third video frame could not be 
nomnally received, a retransmission thereof is per- 
formed in said second or third multiplex frame. 

37. The multiplex transmission method according to 
claim 35, wherein a fixed-length multiplex header at 
least including multiplex information indicating the 
internal multiplex structure and an error detecting 
code for detecting en^or of the multiplex information 
is located at the head of each of said first and sec- 
ond nftultiplex frames, and 

in each of said first and secorxi multiplex 
frames, an ARQ header for each of said first and 
second video frames is stored immediately after 
said multiplex header or at the end of each of said 
first and second multiplex frames, 

wherein when an error is detected in said 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame including said 



ARQ header is read and retransmission control 
thereof is performed. 

38. The multiplex transmission method according to 
B claim 35, wherein a fixed-length multiplex header at 

least including multplex information indicating inter- 
nal multiplex structure and an error detecting code 
for detecting error of the multiplex irtfbrmation is 
located at the head and at the end of each of said 

10 first and second multiplex frames, and 

in each of said first and second multiplex 
frames, an ARQ header for each of said first and 
second video frames is stored immediately after the 
multiplex header located at the head of each of said 

15 first and second multiplex frames or immediately 
before the multiplex header located at tiie end of 
each of said first and second multiplex frames, 

wherein v^en an error is detected in said 
multiplex header and the internal multiplex struc- 

20 ture is not known, tiie video frame Including said 
ARQ header is read and retransmission control 
thereof is performed. 

39. A multiplex fransmission method in which an audio 
25 frame having a frame length of 30 ms and being of 

120-byte cyde when transmitted at 32 Kbps is 
stored, multiplexed and transmitted to a multiplex 
frame having a frame length of 15 ms and being of 
60-byte cyde when transmitted at 32 Kbps, 
30 wherein a first nnultiplex frame containing 

said audio frame and a second multiplex frame con- 
taining none of the audio frame are alternately and 
repeatedly transmitted. 

35 40. The multiplex transmission method aax)rding to 
claim 39, wherein said first multiplex frame contains 
a first video frame which is one complete video 
ARQ frame, together vWth said audio frame, and 
said second multiplex frame contains a sec- 
40 ond video frame which is one connplete video ARQ 
frame having a frame lengtii equal to or larger than 
the sum of the frame length of said audio frame and 
tiiat of said first video frame in said first multiplex 
frame. 

45 

41. The multiplex transmission method according to 
daim 40, wherein when said audio frame is not 
transmitted in a silent period, said first multiplex 
frame is used as a tiiird multiplex frame which forms 

50 a third video frame which is one complete video 
ARQ frame having a frame lengtii equal to tiie sum 
of tiie frame length of said audio frame and that of 
said first video frame, and 

when said third video frame can not be nor- 
55 malty received, a retransmission thereof is per- 
formed in said second or third multiplex frame. 

42. The multiplex transmission method according to 
daim 40, wherein a fixed-length multiplex header at 
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least including multiplex information indicating the 
internal multiplex structure and an en'or detecting 
code for detecting error of the multiplex information 
Is located at the head of each of said first and sec- 
ond multiptex frames, and 5 

in each of said first and second multiplex 
frames, an ARQ header for each of said first and 
second video frames is stored immediately after 
said multiplex header or at the end of each of said 
first and second multiplex frames, 

wherein when an error is detected in said 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame including said 
ARQ header is read and retransmission control 
thereof is performed: /5 

43. The multiplex transmission method according to 
claim 40, wherein a fixed-length multiplex header at 
least including multiplex information indicating the 
internal multiplex structure and an en^or detecting 20 
code for detecting error of the nruiltiplex information 

is located at the head and at the end of each of said 
first and second multiplex frames, 

in each of said first and second multiplex 
frames, an ARQ header for each of said first and 25 
second video frames is stored immediately after the 
multiplex header located at the head of each of said 
first and second multiplex frames or immediately 
before the multiplex header located at the end of 
each of said first and second multiplex frames, 30 

wherein when an error is detected in said 
multiplex header and the internal multiplex struc- 
ture is not known, the video frame including said 
ARQ header is read and retransmission corrtrol 
thereof is performed. 35 

44. A system for multiplexing variable-length data 
employing retransmission to which error correction 
by retransmission control is applied and fixed- 
length data not employing retransmission to which 40 
en-or correction by retransmission control is not 
applied by multiplex frames from a transmitting 
device to a receiving device, 

wherein said transmitting device provides 
said multiptex frame with a fixed length and 4S 
increases/decreases the frame length of a data link 
frame for storing said variable-length data employ- 
ing retransmission depending on the length of said 
multiplexed fixed-length data not employing 
retransmission to transmit the data link frame at a so 
timing always in synchronization with the transmis- 
sion stream of said multiplex frames. 

45. The multiplex transmission system according to 
claim 44, wherein said multiplex frame includes a 55 
fixed-length field, 

said fixed-length field being divided into first 
and second variable-length slots, 

said first variable-length slot having a prede- 



termined fixed length when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a length of zero when no fixed-length 
data not employing retransmission to be multi- 
plexed exists, 

said second variable-length slot having a rel- 
atively short, first length when fixed-length data not 
employing retransmission to be multiplexed exists 
and having a relatively long, second length when no 
fixed-length data not employing retransmission to 
be multiplexed exists, 

wherein said transmitting device transmits 
said fixed-length data not employing retransmission 
in said first variable-length slot and transmits said 
\«riable-length data employing retransmission in 
said second variable-length slot. 

46. The multiplex transmission system according to 
claim 45, wherein said transmitting device has a 
first data link processing block performing transmis- 
sion control of said variable-length data employing 
retransmission only in the form of a fixed-length 
ARQ frame, 

said first data link processing block gener- 
ates a fixed-length ARQ frame in which the data 
length of valid data in said variable-length data 
employing retransmission is increased/decreased 
depending on whether said fixed-length data not 
employing retransmission to be multiplexed exists 
and dummy bits are inserted in the remaining part, 
and 

said transmitting device removes the dumnny 
bits from said ARQ frame to convert the ARQ frame 
into a data link frame of variable-length data 
employing retrar^mission having a frame length 
equal to the length of said second variable-length 
slot, and 

multiplexes the data link frame of saki varia- 
ble-length data employing retransmission and the 
data frame of said fixed-length data not employing 
retransmission to generate said multiplex frame to 
be transmitted to said receiving device. 

47. The multiplex transmission system according to 
claim 46, wherein said receiving device has a sec- 
ond data link processing block performing trar^mis- 
sion control of said variable-length data employing 
retransmission only in the form of a fixed-length 
ARQ frame, 

said receiving device demultiplexes a 
received multiplex frame into the data link frame of 
said variable-length data employing retransmissk)n 
and the data frame of said fixed-length data not 
employing retransmission, 

inserts dummy bits into said demultiplexed 
data link frame of said variable-length data employ- 
ing retransmission depending on whether the 
received multiplex frame includes sab fixed-length 
data not employing retransmission to convert the 
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data link frame of said variable-length data employ- 
ing retransmission into said fixed-length ARQ 
frame, and 

the ARQ frame obtained by said conversion 
is processed by said second data link processing 5 
block. 



mrtted from said first transmission path to said sec- 
ond transmission path, said data not employing 
retransmission in which a transmission en-or has 
occurred on said first transmission path is dis- 
carded at the connection from said first transmis- 
sion path to said second transmission path. 



48. The multiplex transmission system according to 
claim 47, wherein the frame length of said multiplex 
frame is equal to that of said ARQ frame. 

49. A method for performing multiplex transmission 
between a transmitting side and a receiving side by 
using multiplex frames on a composite data trans- 
mission network in which a first netwvork performing 
synchronous communication and a second network 
performing asynchronous communication are con- 
nected to each other. 

wherein multiplex frames containing syn- 
chronization flags at the heads are always transmit- 
ted from said transmitting side to said receiving side 
while the communication is continuing, and 

when a communication is made within said 
first network, frame synchronization established 
prior to the data conrvnunication is kept by using 
said multiplex frames having a fixed frame length, 
without using said synchronization flags. 

50. A method for performing multiplex transmission 
between a transmitting side and a receiving side by 
using multiplex frames in a composite data trans- 
mission network in which a first network performing 
syrKhronous communication and a second network 
performing asynchronous communication are con- 
nected to each other, 

wherein when said second network is inter- 
posed between said transmitting side and said 
receiving side, frame synchronization is acquired 
between said transrrdtting side and said receiving 
side by transmitting multiplex frames containing 
synchronization flags at the heads, and 

v^en making a communication within said 
first networK multiplex frames containing no syn- 
chronization flags are transmitted and frame syn- 
chronization establ^ed prior to the data 
communication is kept by using said multiplex 
frames having a fixed frame length. 

51. A method for multiplexing and transmitting data 
employing retransmission to which error correction 
by retransmission control is appli^ and data not 
employing retransmission to which en<x correction 
by retransmission control is not applied by using 
multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 
tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other, 

wherein when said multiplex frame is trans- 



52. A method for multiplexing and transmitting data 
employing retransmission to which error correction 

10 by retransmission control is applied and data not 
employing retransmission to which error correction 
by retransmission control is not applied by using 
multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 

15 tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other. 

wherein when said multiplex frame is trans- • 
mitted from said first transmission path to said sec- 

20 ond transmission path , 

said data employing retransmission is pro- 
vided with a blockK^oded error correcting code and 
transmitted on said first transmission path, 

at the connection from said first transmission 

25 path to said second transmission patii. said data 
employing retransmission is subjected to en-or cor- 
rection with said error correcting code and then out- 
putted onto said second transmission path with said 
error con-ection code removed tiierefrom, and 

30 retransmission control thereof is performed 

between a transmitting terminal and a receiving ter- 
minal. 

53. A method for multiplexing and transmitting data 
35 employing retransmission to which error correction 

by retransmission control is applied and data not 
employing retransmission to which error correction 
by retransmission control is not applied by using 
multiplex frames on a composite data transmission 
40 path in which a first transmission path with a rela- 
tively high bit rate and a second transmission patii 
witii a relatively low bit rate are connected to each 
other, 

wherein when said multiplex frame is trans- 
45 mitted from said first transmission path to said sec- 
ond transmission path, 

said data employing retransmission is pro- 
vided with a block-coded error correcting code and 
transmitted on said first transmission path. 
so at the connection from said first transmission 

path to said second transmission path, said error 
correcting code is just removed, without error cor- 
rection witii said error correcting code, and then 
said data employing retransmission is outputted 
55 onto said second transmission path, and 

retransmission control thereof is performed 
between a transmitting terminal and a receiving ter- 
minal. 
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54. A method for multiplexing and transmitting data 
employing retransmission to which error correction 
by retransmission control is applied and data not 
employing retransmission to which error correction 
by retransmission control is not applied by using 
multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 
tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other, 

wherein when said multiplex frame is trans- 
mitted from said first transmission path to said sec- 
ond transmission path, 

said data employing retransmission is con- 
voiutional-coded with a certain code rate Rc and 
transmitted on said first transmission path. 

at the connection from said first transmission 
path to said second transmission path, said convo- 
lutional coding is decoded and then said data 
employing retransmission is outputted onto said 
second transmission path, and 

its retransmission control is performed 
between a transnnitting terminal and a receiving ter- 
minal. 

55. The multiplex transmission method according to 
claim 54. wherein when Rv expresses a value 
obtained from the bit rate Vf of said first transmis- 
sion path and the bit rate Vs of said second trans- 
mission path, { Rv=(Vf-Vs)M }, 

Lm expresses the frame length of said multi- 
plex frame, 

Ld expresses the data length of said data 
employing retransmission subjected to said 
convolutional coding, and 
Lo expresses the length of overhead newly 
added at the time of transmission on said sec- 
ond transmission path, then 
said certain code rate Rc is calculated as 

Rc=ljd/{Lm • Rv-Lo+Ld). 

56. A method for multiplexing and transmitting data 
employing retransnnission to which error con'ection 
by retransmission control is applied and data not 
employing retransmission to which error conrection 
by retransmission control is not applied by using 
multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 
tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other, 

wherein when said multiplex frame is trans- 
mitted from said first transmission path to said sec- 
ond transmission path, 

said data not employing retransmission is 
convdutional-coded with a certain code rate Rc 
and transmitted on said first transmission path, and 



at the connection from said first transmission 
path to said second transmission path, said convo- 
lutional coding is decoded and then said data not 
employing retransmission is outputted onto said 
5 second transmission path. 

57. The multiplex transmission method according to 
claim 56, wherein when Rv expresses a value 
obtained from the bit rate Vf of said first transmis- 
10 sion path and the bit rate Vs of said second trans- 
mission path, {Rv=(Vf-Vs)/Vf }, 

Lm expresses the frame length of said multi- 
plex frame, 

15 Lxl expresses the data length of said data not 

employing retransmission subjected to said 
convolutional coding, and 
Lo expresses the length of overhead newly 
added at the time of transmission on said sec- 

20 ond transmission path, then 

said certain code rate Rc is calculated as 

Rc=Ld/(LjTi • Rv-LchUJ). 

25 58. A method for multiplexing and transmitting data 
employing retransmission to which error correction 
by retransmission control is applied arxi data not 
employing retransmission to which error correction 
by retransmission control is not applied by using 

30 multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 
tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other, 

35 wherein when said multiplex frame is trans- 

mitted from said first transmission path to said sec- 
ond transmission path, 

said data not employing retransmission is 
provided with a block-coded error correcting code 

40 and transmitted on said first transmission path, and 
at the connection from said first transmission 
path to said second transmission path, said data 
not employing retransmission is subjected to en'or 
correction with said error correcting code and then 

45 outputted onto said second transmission path with 
said en'or connecting code removed therefrom. 

59. A method for multplexing data employing retrans- 
mission to which error correction by retransmission 

so control is applied and data not employing retrans- 
mission to which error correction by retransmission 
control is not applied by using multiplex frames on a 
composite data transmission path in which a first 
transmission path with a relatively high bit rate and 

55 a second transmission path with a relatively tow bit 
rate are connected to each other, 

wherein when said multiplex frame is trans- 
mitted from said first transmission path to said sec- 
ond transmission path. 
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said data not employing retransnnission is 
provided with a block-coded error correcting code 
and transmitted on said first transmission path, and 

at the connection from said first transmission 
path to said second transmission path, said error 5 
correcting code is just removed, without error cor- 
rection with said error correcting code, and then 
said data not employing retransmission is outputted 
onto said second transmission path. 

10 

60. A method for multiplexing and transmitting data 
employing retransmission to which en'or correction 
by retransmission control is applied and data not 
employing retransmission to which error correction 

by retransmission control is not applied by using is 
multiplex frames on a composite data transmission 
path in which a first transmission path with a rela- 
tively high bit rate and a second transmission path 
with a relatively low bit rate are connected to each 
other, 20 

wherein when said multiplex frame is trans- 
mitted from said first transmission path to said sec- 
ond transmission path. 

said data employing retransmission is con- 
volutional-coded with a certain code rate Rc and 2s 
transmitted on said first transmission path, and 

at the connection from said first transmission 
path to said second transmission path, said convo- 
lutional coding is decoded and then said data 
employing retransmission is outputted onto said 30 
second transmission path, and 

its retransmission control is performed 
between a transmitting terminal and a receiving ter- 
minal, and 

said data not employing retransmission is 35 
convolutional-coded with a certain code rate Rc 
and transmitted on said first transmission path, and 

at the connection from said first transmission 
path to said second transmission path, said convo- 
lutionat coding is decoded and then said data not 40 
employing retransmission is outputted onto said 
second transmission path. 

61. The multiplex transmission method according to 
claim 60, wherein when Rv expresses a value 45 
obtained from the bit rate Vf of said first transmis- 
sion path and the bit rate Vs of said second trans- 
mission path, {Rv=(Vf-Vs)/Vf }, 

Lm expresses the frame length of said multi- so 
plex frame, 

Ld expresses the total of the data lengths of 
said data employing retransmission and said 
data not employing retransmission subjected to 
said convolutional coding, and ss 
Lo expresses the length of overhead newly 
added at the time of transmission on said sec- 
ond transmission path, then 
said certain code rate Rc is calculated as 
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Rc=ljd/(ljn - Rv-Lo+Ld). 

62. A method for multiplexing and transmitting variable- 
length data employing retransmission to which 
error connection by retransmission control is applied 
and fixed-length data not enploying retransmission 
to which en-or correction by retransmission control 
is not applied in multiplex frames, wherein 

said multiplex frames have a fixed length and 
a data link frame for storing said variable-length 
data employing retransmission has a frame length 
increased/decreased depending on the length of 
said fixed-length data not employing retransmission 
to be multiplexed so that said data link frame is 
transmitted at timing kept in synchronization with 
the transmission stream of said multiplex frames. 

63. A storage medium for storing software program to 
be executed in order to multiplex and transmit vari- 
able-length data employing retransmission to which 
error connection by retransmission control is applied 
and fixed-length data not emptying retransmission 
to which en^or correction by retransmission control 
is not applied in multiplex frames, wherein 

said software program fix a length of said 
multiplex frame length and increase/decrease the 
length of data link frame length for storing said var- 
iable-length data employing retransmission 
depending on the length of said fixed-length data 
not emptying retransmission to be multiplexed so 
that the data link frame is transmitted at timing kept 
in synchronization, for each said multiplex frame, 
with the transmission stream of said multiplex 
frames. 

64. The storage medium according to claim 63. 
wherein when fixed-length data not employing 
retransmission to be multiplexed exists, a basic 
multiplex frame stream including at least one first 
nrujltiplex frame in which said fixed-length data not 
employing retransmission is multiplexed and at 
least one second multiplex frame in which said 
fixed-l^gth data not employing retransnnision is not 
multiplexed is repeatedly transmitted. 

65. A storage medium for storing software program to 
be excuted in order to absorb the audio jitter on the 
side receiving multiplex frames when a low-bitrate- 
coded N-byte (N is a positive integer) audio frame 
generated at a rate of one frame per an F-byte (F is 
a positive integer and N<F) transmission time is 
stored and multiplexed in an M-byte (M is a positive 
integer and M<F) fixed-length multiplex frame and 
when at most J-byte (J is a positive integer) audio 
jitter is generated among the audio fields of the nruil- 
tiplex frames because F/M is not an integer, 

wherein said software program comprises 

a first program step for identif ing a reference 
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audio frame separated from the next audio 
frame at a minimum interval among the audio 
frames on a stream of the transmitted multiplex 
frame, and 

a second program step for outputting the fol- s 
lowing audio frames at the rate of one frame 
per the F-byte transmission time using the tim- 
ing at which said reference audio frame Is out- 
putted as a reference, 

wherein different multiplex information 10 
are defined for n multiplex frames containing 
multiplexed audio frames in a basic multiplex 
frame stream formed of m multiplex frames, 
where n and m are integers and 
n • F=m • M , and 15 
said reference audio frame is identified on the 
basis of said multiplex information at said first 
program step. 

66. A storage medium for storing software program to 20 
be excuted in order to absorb the audio jitter on the 
side transmitting multiplex frames when a low- 
bitrate-ooded N-byte (N is a positive integer) audio 
frame generated at a rate of one frame per an F- 
byte (F is a positive integer and N<F) transmission 2s 
time is stored and multiplexed in an M-byte (M is a 
positive integer and M<F) fixed-length multiplex 
frame and when at most J-byte (J is a positive inte- 
ger) audio jitter is generated among the audio fields 

of the multiplex frames because F/M Is not an inte- 30 
ger, 

wherein when the time from the start of a 
coding operation of an audio coder for coding audio 
data to the start of output of the first audio frame 
from the audio coder is taken as T and the time 3S 
required from the start of output of the first audio 
frame from said audio coder to the start of output of 
the multiplex frame containing said first audio frame 
is taken as a, 

using as a reference the timing of the start of 4o 
output of the multiplex frame storing a reference 
audio frame separated from the next audio frame at 
a minimum inten^al in the audio frames on the 
transmitted multiplex frame stream, the coding 
operation of said audio coder is started preceding 4s 
the reference timing at least for the total time of 
(T+J+a). 

wherein different multiplex information are 
defined for n multiplex frames containing multi- 
plexed audio frames in a basic multiplex frame so 
stream formed of m multiplex frames, where n and 
m are integers and n • F=m • M , and 

said reference audio frame is identified on 
the basis of said multiplex information by said soft- 
ware prt^ram. 55 

67. A storage medium for storing software program to 
be executed in order to store in multiplex frames, 
multiplex and tranmit a first data frame provided 



with a frame nunt)er and an error detecting code 
and second to kth data f rame(s) (k is an integer of 2 
or larger), 

wherein said multiplex frame is further pro- 
vided with multiplex information indicating the struc- 
ture of the multiplexing of said first data frame and 
said second to kth data frame(s). and 

wherein said software program has 

said transmitting side locate the frame 
number of said first data frame always in a fixed 
position in said multiplex frame and 

when detecting presence of an error in said 
multiplex information in the multiplex frame 
received, said receiving side read the frame 
number of said first data frame from the fixed posi- 
tion in the multiplex frame and transmit a retrans- 
mission request for the read frame number to said 
transmitting side. 
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FIG. 14 
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FIG. 19 



MULTIPLEX 
INFORMATION (HC) 





MULTIPLEX STRUCTURE 




AUDIO 25 BYTES, VIDEO 51 BYTES 




AUDIO 25 BYTES, VIDEO 51 BYTES 




VIDEO 76 BYTES 




DATA 25 BYTES, VIDEO 51 BYTES 



60 



EP0 806 873 A2 



FIG. 2 0 
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FIG. 2 1 
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FIG. 2 7 (a) 



FIG. 2 7 (b) 
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